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FINAL  TECHNICAL  REPORT 


07/1/04-09/15/06 

National  Center  for  Collaboration  in  Medical  Modeling  and  Simulation 
Eastern  Virginia  Medical  School 
and 

Virginia  Modeling,  Analysis  and  Simulation  Center 

Background  -  The  Office  of  Naval  Research  Human  Systems  Department  (HSD)  is  comprised 
of  two  divisions:  Medical  and  Biological  Science  and  Technology  and  Cognitive,  Neural  and 
Social  Science  and  Technology.  The  HSD  supports  basic  and  applied  research,  and  advanced 
technology  development,  leading  to  applications  for  the  Departments  of  the  Navy  and  Defense, 
and  U.S.  industry.  The  HSD  is  committed  to  an  active  exploration  program  at  the  leading  edges 
of  medical  science,  human  performance,  biotechnology,  training  and  human  factors,  neural 
information  processing,  and  biorobotics.  Programs  supported  by  the  divisions  range  from 
molecular  biology  to  the  development  of  advanced  medical  therapies  for  saving  lives  and 
strategies  for  preserving  a  healthy  and  fit  fighting  force.  As  part  of  this  mission,  the  HSD  had 
identified  a  significant  problem  that  negatively  impacts  both  the  quantity  and  quality  of  medical 
care  available  to  the  U.S.  Navy  and  Marine  Corps  in  times  of  military  conflict  and  in  peace  time 
in  CONUS  and  other  facilities.  The  crux  of  the  problem  is  the  need  to  train  medical  personnel,  in 
relatively  large  quantities,  to  effectively  deal  with  trauma-related  combat  casualties,  and 
casualties  due  to  chemical  or  biological  agents. 

Objective  -The  objective  was  to  develop  and  integrate  modeling  and  simulation  technology  into 
medical  education,  patient  care  and  research,  with  the  goal  of  transforming  the  way  in  which 
medical  personnel  are  educated  and  trained  and  leveraging  the  investment  made  to  develop  these 
technologies  by  training  more  personnel,  in  less  time,  with  more  effective  results;  to  develop 
modeling  and  simulation  solutions  to  the  planning,  training,  and  execution  of  medical  responses 
to  events  that  produce  mass  casualties.  In  addition,  this  project  was  to  (1)  address  the  care  of 
military  personnel  and  their  dependents  in  non-combat-related  diagnosis  and  treatment,  (2)  apply 
modeling  and  simulation  to  the  better  prepare  the  military  and  the  nation  at  large  to  respond 
medically  to  events  that  produce  mass  casualties,  and  (3)  to  influence  the  training  of  medical 
personnel  through  determining  the  efficacy  of  available  simulators  and  their  role  in  the  medical 
curriculum. 

General  -  The  Contractor  was  to  provide  all  necessary  personnel,  facilities,  and  equipment  necessary 
to  accomplish  the  work  described  in  the  following  tasks.  These  services  included,  but  were  not 
limited  to:  validating  medical  simulators,  developing  new  technologies,  integrating  simulators 
and  simulations  into  medical  curricula,  and  developing  regional  models  to  assist  in  planning, 
training,  and  decision-support  in  the  event  of  an  incident  producing  mass  casualties. 

Requirement  -  The  following  specific  tasks  were  to  be  accomplished  during  the  period  of 
performance. 

Task  1  -  Simulator  Validation 

1.1  Conduct  a  validation  experiment  of  the  catheter  insertion  simulator  using  EVMS  medical, 
EVMS  physician’s  assistant,  and  ODU  psychology  students  (Completed) 


Two  experiments  were  conducted  comparing  the  CathSim®  virtual  reality  simulator  and 
simulated  limbs  for  training  phlebotomy.  In  the  first  experiment,  medical  students  completed  two 
one-hour  training  sessions  using  one  of  the  two  methods.  Afterward,  the  students  had  their 
performance  assessed  with  both  simulated  limbs  and  actual  patients.  The  results  showed  that 
students  trained  using  the  simulated  limbs  as  compared  to  the  CathSim®  system  achieved  higher 
scores  on  the  posttest  and  on  the  field  test  with  actual  patients.  In  the  second  study,  graduate 
students  practiced  with  the  CathSim®  system  for  five  hours.  The  results  showed  a  significant 
improvement  in  scores  from  the  pretest  to  posttest  for  these  students;  however,  even  after 
extended  practice  their  final  scores  did  not  differ  from  those  of  the  medical  students  in  the  first 
experiment.  Critical  differences  in  design  characteristics  between  the  two  systems  may  explain 
these  findings.  Although  a  clear  advantage  for  simulated  limbs  over  the  CathSim®  system  was 
observed,  neither  system  faithfully  represents  all  of  steps  needed  to  perform  the  procedure.  Thus, 
training  decisions  may  require  one  to  consider  the  unique  advantages  afforded  by  each  system. 
(See  Deliverable  RIB  -  Appendix  A) 


1.2  Conduct  a  validation  experiment  of  the  colonoscopy  simulator  using  EVMS  surgical  and 
ODU  psychology  students.  (Partially  Completed) 

EVMS  surgical  students  and  ODU  psychology  students  all  received  initial  training  on  the 
colonoscopy  simulator.  All  psychology  students  (n=8)  completed  10  hours  of  training.  None  of 
the  surgical  students  completed  training.  The  results  for  the  psychology  students  showed 
significant  decreases  in  the  time  needed  to  complete  the  procedure  and  time  in  red-out  (an 
accuracy  measure)  after  10  hours  of  practice.  Unfortunately,  without  data  from  the  surgical 
residents,  there  is  no  basis  for  comparison. 


1.3  Provide  support  for  a  validation  experiment  of  an  arthroscopy  simulator  using  surgical  staff  at 
Portsmouth  Naval  Hospital  and  ODU  psychology  students.  (Partially  Completed) 

Logistical  support  and  student  help  were  provided  to  staff  at  the  Portsmouth  Naval  Hospital  to 
conduct  their  evaluation  of  the  arthroscopy  simulator.  The  first  phase  of  the  virtual  arthroscopy 
project  consisted  of  the  development  and  validation  of  a  scoring  system  for  diagnostic 
arthroscopy.  This  first  study  was  a  blinded  randomized  prospective  study.  (See  Deliverable  R1 A 
-  Appendix  A) 


1 .4  Provide  human  factors  input  into  the  development  of  a  high  performance  skill  medical 
simulators.  (Completed) 

Human  factors  principles  were  applied  in  the  design  and  construction  of  the  wound  debridement 
system.  (See  Deliverable  RIB,  R1C  -  Appendix  B) 


1 .5  Perform  literature  review  of  research  on  high  performance  skill  acquisition.  (Completed) 

A  literature  review  examining  recent  research  on  the  efficacy  of  simulators  for  training  surgical 
skills  was  undertaken.  In  surgery,  the  most  common  type  of  simulators  target  laparscopic 
procedures  and  the  Minimally  Invasive  Surgical  Trainer  Virtual  Reality  (MIST  VR)  is  the  most 
widely  studied  laparoscopic  simulator.  Despite  its  research  exposure,  however,  few  studies  have 
demonstrated  the  transfer  of  skills  learned  on  the  MIST  VR  to  procedures  performed  on  genuine 


patients.  Further,  most  of  the  research  to  date  is  not  tied  to  any  theories  of  learning.  The  review 
describes  current  research  on  laparoscopic  training  with  simulators  and  discusses  the  need  for 
theories  of  learning  and  skill  acquisition  in  future  research.  (See  Appendix  C) 


1.6  Organize  professional  meetings  to  address  current  state  of  modeling  and  simulation  methods 
and  technology  and  its  applicability  to  medicine.  (Not  completed) 

The  prospect  for  hosting  a  professional  meeting  on  simulation  in  medicine  that  did  not  duplicate 
existing  meetings  was  examined.  Several  new  forums  for  the  exchange  of  technical  information 
on  this  topic  have  emerged  and  a  decision  was  made  not  to  proceed  at  this  time. 


1 .7  Continue  a  research  program  to  study  the  practice  of  medicine/surgery  (both  research  and 
application  issues).  (Completed) 

Research  continued  throughout  the  duration  of  the  project.  A  report  on  key  user-based  research 
issues  was  prepared  (see  Appendix  D).  In  addition,  a  competency  modeling  approach  was 
followed  to  understand  excellence  in  surgery  (see  Appendix  E). 


1.8  Continue  work  on  surgical  skills  under  combat  study.  (Completed) 

A  group  of  medical  students  were  taught  to  perform  a  tube  thoracostomy  on  a  mannequin 
simulator  in  a  traditional  medical  school  setting  and  demonstrate  competency  in  that 
environment.  Immediately  afterward  and  again  four  months  later,  they  performed  the  procedure 
in  a  CAVE  virtual  environment  running  a  combat  simulation  under  day  and  nighttime  lighting 
conditions.  The  results  showed  that  accuracy  suffered  in  the  simulation.  Participants  also  needed 
more  time  to  perform  the  procedures  under  the  nighttime  conditions.  Further,  performance  did  not 
change  over  the  retention  interval.  The  results  suggest  that  although  the  surgical  skills  acquired 
by  students  in  a  traditional  medical  school  setting  are  robust,  they  may  still  be  compromised 
under  hazardous  conditions.  More  important,  these  findings  show  that  virtual  environments  can 
provide  a  safe  and  effective  venue  for  military  medical  personnel  to  train  for  dangerous  duty.  The 
last  technical  report  on  this  project  is  included  in  Appendix  F. 


1.9  Continue  work  on  simulated  patient  assessment.  (Completed) 

Human  factors  design  and  support  were  provided  on  a  project  examining  the  efficacy  of  training 
with  standardized  patients.  Residents  in  the  EVMS  dermatology  program  saw  standardized 
patients  addressing  the  same  4  skin  disorders  in  the  Skills  Center  and  in  the  field  in  a  genuine 
clinical  setting.  Overall,  the  results  showed  that  field  test  scores  were  lower  than  the  OSCE  scores 
from  the  Skills  Center  and  suggest  classical  observational  effects.  The  scores  for  those  residents 
who  were  first  observed  in  the  field  improved  by  about  10  points  when  later  observed  in  the 
OSCE  environment.  By  contrast,  the  scores  for  those  residents  who  were  first  observed  in  the 
OSCE  environment  declined  by  about  4.5  points  when  later  observed  in  the  field.  In  addition,  the 
field  test  scores  were  similar  irrespective  of  when  they  were  observed. 


1.10  Continue  work  on  suturing  skills  assessment.  ( Completed) 


A  workshop  for  teaching  medical  students  fundamental  suturing  skills  was  conducted.  Students 
were  taught  3  basic  stitches  and  allowed  to  practice  them  on  pads  of  simulated  skin.  The 
workshop  was  only  offered  once. 


Task  2  -  In  the  area  of  Technology  Development,  the  Subcontractor  shall: 

2.1  Maintain  the  existing  inventory  of  DOD  medical  modeling  and  simulation  activities,  current 
projects  and  points  of  contact.  (Completed) 

Began  initial  inventory  of  medical  M&S  technology.  This  material  was  subsumed  within  the 
medical  M&S  database  project. 

2.2  Develop  collaborative  mechanisms  for  medical  modeling  and  simulation  activities 
nationwide.  (Completed) 

Collaborations  were  developed  and/or  maintained  by  1)  participating  in  the  Advanced  Initiatives 
in  Medical  Simulation  delegation  to  the  U.S.  Capitol,  Washington,  D.C.  and  facilitating 
demonstrations  of  medical  simulation  technology  designed  to  promote  patient  safety  in  the 
Dirksen  Senate  Office  Building  in  2004,  2005,  and  2006;  and  2)  serving  on  the  Medical 
Simulation  Task  Force  of  the  Society  for  Interventional  Radiologists. 


2.3  Demonstrate  the  value  of  using  medical  and  surgical  simulators  in  validation  and  training 
transfer  studies.  (Completed) 

See  Task  1.1  above 

2.4  Survey  existing  tissue  models  and  simulator  technologies  and  begin  selective  development  of 
medical  and  surgical  simulation.  (Completed) 

Many  tissue  models  were  identified  and  studied  including  mass-spring  and  finite  element 
viscoelastic  models.  The  impact  of  computing  technology  was  considered  in  evaluating  the 
different  models.  In  general,  finite  element  models  provided  the  best  fidelity  in  terms  of 
modeling  the  viscoelastic  properties  of  tissue.  Mass  spring  models  lack  the  necessary  fidelity  and 
are  difficult  to  tune. 

At  the  current  time,  finite  element  models  are  unsuitable  for  deployment  with  the  current  desktop 
computing  technology  because  these  models  cannot  provide  the  real-time  fidelity  required  for 
surgical  skills  training.  Mass-spring  models  more  readily  provide  real  time  performance,  but  not 
at  the  same  fidelity.  Weighing  the  trade-offs,  the  team  decided  to  implement  the  prototype  using 
a  simple  mass  spring  tissue  model,  and  concurrently  study  the  possibilities  offered  by  hybrid 
models.  A  bibliography  of  tissue  models  described  in  the  literature  appears  in  Appendix  G. 


Task  3  -  In  the  area  of  Simulators  and  Curriculum  Integration,  the  Subcontractor  shall: 

3.1  Develop  detailed  training  simulator  requirements  based  on  the  identified  use  cases.  These 
requirements  will  form  the  basis  for  simulator  design  and  functionality.  (Completed) 


Detailed  training  simulator  requirements  were  developed  for  the  wound  debridement  procedure. 
Wound  debridement  is  an  example  of  a  basic  open  surgical  procedure  requiring  the  trainee  to 
follow  a  specific  sequence  of  steps  in  order  to  correctly  complete  the  procedure.  The  team  met 
with  surgeons  to  determine  the  necessary  educational  content,  determine  the  precise  sequence  to 
follow  to  complete  the  procedure,  and  the  requirements  for  each.  A  more  in-depth  summary  is 
included  in  Appendix  B. 


3.2  Based  on  the  training  simulator  requirements,  develop  a  conceptual  model  of  high 
performance  skill  medical  training  simulators.  (Completed) 

A  conceptual  model  for  a  wound  debridement  trainer  was  developed  that  incorporated  several 
training  modes  including  didactic  and  simulated  training  experiences.  The  trainer  is  described  in 
more  depth  in  Appendix  B. 


3.3  Develop  a  research  plan  that  will  support  the  high  performance  skill  medical  training 
simulator  development.  This  effort  will  include  both  the  technologies  required  to  support 
the  medical  training  simulator  and  the  technology  and  methodology  to  support  effective 
training  tasks  and  transference  of  those  to  the  real  world.  (Completed) 

A  plan  for  assessing  performance  with  medical  simulators  was  adapted  from  aviation  training 
transfer  paradigms.  (See  Appendix  H) 

3.4  Identify  and  explore  haptics,  auditory,  simulation  and  visualization  technologies  that  are 
available  to  implement  high  performance  skill  medical  training  simulators.  This  effort 
will  include  identification  of  a  core  set  of  technologies  that  are  readily  available  and 
could  be  used  to  support  a  medical  training  simulator  in  the  first  quarter  of  calendar  year 
2005.  (Completed) 

The  technologies  identified  were  assessed  and  used  to  support  the  development  of  the  wound 
debridement  and  augmented  standardized  patient  trainers.  Core  technologies  for  haptics 
(SensAble  Technologies,  Phantom  Premium),  auditory  (electronic  stethoscopes,  speech 
recognition,  speech  synthesis),  and  simulation  (Derivative,  literature  survey)  were  reviewed. 
Decoupled  from  other  technologies,  general  visualization  technology  was  viewed  to  be  generally 
adequate  for  most  visual  processing  to  the  support  medical  skills  training  provided.  Special 
purpose  assemblies,  such  as  the  Reachln  Display,  that  integrates  haptic  and  visual  experiences, 
were  viewed  to  be  important  in  creating  an  effective  training  simulation. 


3.5  Based  on  the  conceptual  model,  develop  a  high  performance  skill  medical  training 

simulator  system  design  that  will  meet  the  identified  simulator  requirements.  (Completed) 

A  prototype  wound  debridement  trainer  was  developed.  The  trainer  is  described  in  more  depth  in 
Appendices  B,  I,  and  J. 
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ABSTRACT 


Background:  The  knee  arthroscopy  scoring  system  is  a  tool  designed  to  objectively 
measure  technical  performance  and  to  distinguish  the  novice  from  the  proficient  surgeon 
who  requires  minimal  supervision.  The  purpose  of  this  study  is  to  validate  this  scoring 
system  assessing  resident  proficiency  during  diagnostic  arthroscopy. 

Methods:  Randomized  prospective  blinded  study  that  included  residents  PGY  1-5  (n=20) 
and  staff  surgeons  (n=10).  Subjects  were  randomized  and  performed  a  diagnostic 
arthroscopy  on  a  cadaver  knee.  Subjects  were  evaluated  both  on  completeness  of  exam 
and  time  taken  to  complete  the  arthroscopy.  The  examiner  viewed  the  arthroscopy  from  a 
remote  location  and  was  blinded  to  the  level  of  training  of  the  subjects.  During  the 
arthroscopy,  24  areas  required  assessment  to  score  75  points.  An  additional  25  points 
were  awarded  depending  upon  the  time  it  took  to  complete  the  arthroscopy.  A  maximum 
of  100  points  were  available. 

Results:  A  total  of  30  subjects  were  enrolled  and  divided  into  three  groups:  Group  1 
(PGY-1/2  n=12),  Group  2  (PGY-3/4/5  n=8),  Group  3  (Staff  n=10).  Group  1  performance: 
average  total  score  27.9,  average  time  to  complete  scope  11.9  min,  average  number  of 
structures  not  examined  8.67.  Group  2  performance:  average  total  score  73.8,  average 
time  8.2  min,  average  structures  not  examined  1.75.  Group  3  performance:  average  total 
score  100,  average  time  4.6  min,  average  structures  not  examined  0.  Statistically 
significant  differences  were  noted  for  the  total  score  of  all  the  groups  (p  <  0.001)  and  for 


the  time  of  arthroscopy  (p  <  0.001).  For  items  not  examined,  Groups  2  and  3  were 
equivalent  and  both  had  a  significant  difference  compared  to  Group  1  (p  <  0.001).  Using 
this  scoring  system,  staff  surgeons  consistently  scored  100  points  and  were  more  efficient 
and  complete  compared  to  residents.  In  addition,  more  experienced  senior  residents 
scored  higher  than  junior  residents. 

Conclusions:  The  knee  arthroscopy  scoring  system  is  a  valid  method  to  distinguish 
between  the  novice,  intermediate,  and  expert  in  diagnostic  arthroscopy.  Utilizing  this 
scoring  system,  we  can  now  objectively  monitor  a  resident’s  progress  in  diagnostic 
arthroscopy  and  identify  individuals  that  may  need  additional  training  prior  to  performing 
arthroscopy  under  limited  supervision. 


Level  of  Evidence:  I 


ARTHROSCOPIC  SKILLS  ASSESSMENT  FORM  -  KNEE 


Figure  1 
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Figure  2 
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these  physical  models  have  evolved  into  mannequin-based 
simulator  systems  coupled  with  computers  that  provide 
dynamic  physiologic  responses  to  treatments.1’1  Within  the 
past  5  years,  an  alternative  form  of  simulation  technology 
also  has  become  viable.  Medical  virtual  reality  (VR)  sim¬ 
ulators  use  computer  systems  combining  visual,  auditory, 
and  haptic  displays  that  enable  individuals  to  see,  hear,  and 
touch  objects  in  virtual  space.  Computer  models  generate 
a  patient’s  anatomy  and  physiology,  and  the  user  interacts 
with  the  virtual  patient  via  computer  interface  devices. 

Medical  simulators  offer  many  benefits.  They  allow 
medical  personnel  to  train  on  devices  instead  of  practicing 
on  patients.  Thus,  trainees  can  learn  from  their  mistakes 
without  putting  patients  at  risk.  Simulators  also  can  pro¬ 
vide  trainees  with  an  unlimited  number  of  trials  to  acquire 
skills.  Moreover,  they  offer  an  environment  for  the  training 
of  specific  skills  in  the  absence  of  uncontrollable  environ¬ 
mental  influences  and  patient  interactions.  Furthermore, 
simulators  help  to  decrease  dependence  on  animals  and 
cadavers  for  practicing  procedures. 

There  also  are  advantages  unique  to  medical  VR  sim¬ 
ulators.  First,  because  the  user  interacts  with  a  virtual 
patient,  there  is  no  need  for  expendable/replaceable  parts 
and  supplies  common  to  many  mannequin-based  systems. 
Second,  most  VR  simulators  allow  students  to  interact  with 
the  system  and  receive  objective  performance  feedback  in 
real  time.3  Third,  virtual  technology  offers  a  more  cost- 
effective  way  to  create  a  wide  range  of  individual  patient 
characteristics  and  pathology,  thereby  allowing  students 
to  be  more  readily  exposed  to  rare  or  infrequent  condi¬ 
tions.  Currently,  a  variety  of  commercially  available  VR 
simulators  address  specific  procedures  including  laparo¬ 
scopic  skills,  arthroscopic  knee  surgery,  endoscopy, 
ophthalmology,  and  sinus  surgery.4 


device  to  insert  the  needle.  The  system  records  many  per¬ 
formance  metrics,  including  procedure  success,  angle  of 
needle  insertion,  occurrence  of  hematoma,  session  time, 
tourniquet  placement  time,  and  a  pain  factor.  In  addition, 
two  visual  training  aids  allow  users  to  see  either  a  repre¬ 
sentation  of  the  needle  entering  the  arm  from  a  side  view 
or  a  “transparent”  view  of  the  underlying  vascular  system. 


•  RESEABdtjT  f 

To  date,  only  a  few  published  studies  have  examined  the 
effectiveness  of  the  CathSim  system  for  teaching  IV  pro¬ 
cedures.  Reznek,  Rawn,  and  Krummel6  found  that  the 
CathSim  system  could  distinguish  among  first-year  med¬ 
ical  students,  third-year  medical  students,  and  surgical  res¬ 
idents.  Several  investigators  have  compared  the  CathSim 
system  and  plastic  arms  used  for  training.  Better  success 
rates  were  observed  for  the  trainees  who  used  the  plastic 
arms.7’8  For  instance,  Scerbo  et  a l9  found  that  students 
benefited  from  training  that  used  both  methods,  but 
those  who  trained  with  plastic  arms  showed  the  greatest 
gains  in  improvement.  More  recently,  Bowyer  et  al’°  com¬ 
pared  simulated  limbs  with  the  CathSim  system  and  the 
Virtual  IV  system  by  Laerdal  Medical,  Inc.  (Wappingers 
Falls,  NY)  and  found  some  evidence  for  greater  perfor¬ 
mance  gains  with  the  Virtual  IV  system. 

Collectively,  the  findings  from  the  studies  examining  the 
CathSim  VR  simulator  show  that  it  is  only  marginally 
effective  in  comparison  with  plastic  arms.  It  is  important 
to  note  that  all  the  aforementioned  research  examined 
the  cannulation  procedure.  None  of  the  studies  addressed 
phlebotomy. 


Recently,  a  VR  simulator  was  developed  for  intravenous 
(IV)  procedures.5  The  CathSim  system  (Immersion  Med¬ 
ical,  Inc.,  Gaithersburg,  Md)  provides  training  on  a  sub¬ 
set  of  vascular  access  procedures,  including  catheter 
insertion  (cannulation)  and  phlebotomy.  The  physical 
system  consists  of  a  personal  computer  accompanied  by 
their  AccuTouch  haptic  feedback  device  with  six  degrees 
of  freedom  that  simulates  the  needle  and  hub  and  a  vinyl 
strip  that  simulates  the  skin.  The  AccuTouch  device 
enables  trainees  to  experience  haptic  force  feedback 
when  inserting  a  needle  into  the  simulated  skin  and  vein. 
The  system  also  includes  a  variety  of  unique  case  patient 
training  scenarios  and  tutorial  video  clips  to  familiarize 
students  with  many  procedural  details. 

The  user  operates  the  system  by  making  selections  with 
the  computer’s  mouse  and  manipulating  the  AccuTouch 


•  PHLEBOTOMY 


Although  both  cannulation  and  phlebotomy  can  be 
considered  vascular  access  procedures,  Ernst  and  Ernst11 
argue  that  they  are  entirely  different  procedures.  Whereas 
the  objective  of  cannulation  is  to  insert  a  needle  into  the 
patient’s  arm  for  the  introduction  of  fluids,  the  purpose 
of  phlebotomy  is  to  access  a  patient’s  vein  and  collect  a 
high-quality  blood  specimen  that  can  be  correctly  identi¬ 
fied  and  safely  transported  for  laboratory  analysis.11 

The  need  for  phlebotomy  training  is  paramount.  A 
poorly  performed  phlebotomy  procedure  can  result  in 
traumatic  punctures,  repeated  punctures,  subcutaneous 
hemorrhages,  hematomas,  chronic  pain  from  punctures 
that  pass  through  a  nerve,  and  even  concussions  and  bone 
fractures  that  occur  during  fainting  spells  of  patients  left 
unattended.11’13  An  improperly  performed  procedure  also 
can  generate  erroneous  laboratory  results  that  either  leave 
existing  conditions  undetected  or  indicate  the  presence  of 
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nonexisting  conditions  that  subject  the  patient  to  addi¬ 
tional  unnecessary  tests,  procedures,  and  all  the  associated 
risks.14 

Besides  patient  safety  issues,  there  are  factors  that  affect 
the  clinician’s  ability  to  perform  the  procedure.  For  exam¬ 
ple,  a  patient’s  age,  size,  obesity,  general  health,  and  per¬ 
sonal  habits  (eg,  smoking  or  intravenous  drug  use)  affect 
not  only  the  ease  with  which  the  vein  can  be  accessed,  but 
also  the  choice  of  vein.  Differences  in  specimen  collection 
equipment  and  materials  also  affect  how  the  procedure 
is  performed  and  the  order  in  which  different  collection 
tubes  are  to  be  filled.15 

Furthermore,  the  phlebotomist  also  can  be  at  risk.16 
Phlebotomy  involves  a  variety  of  unique  safety  issues  con¬ 
cerning  the  practice  of  collecting  multiple  blood  samples 
during  a  single  procedure  and  the  use  of  sharps  disposable 
containers.  Moreover,  the  National  Institute  of  Occupa¬ 
tional  Safety  and  Health  (NIOSH)  estimates  that  600,000 
to  800,000  needlestick  injuries  occur  in  healthcare  set¬ 
tings  each  year,17  exposing  healthcare  providers  to  as 
many  as  20  different  blood-borne  diseases  including  HIV, 
hepatitis  B,  and  hepatitis  C  viruses.11 

Finally,  only  1  state,  California,  has  a  licensure  require¬ 
ment  for  phlebotomists.  Most  states  do  not  even  have 
certification  requirements.  Moreover,  there  are  no 
performance-based  standards  for  assessing  the  phlebotomy 
skills  of  healthcare  workers.  The  common  practice  is  for 
more  advanced  phlebotomists  to  observe  a  trainee  over 
a  designated  period  or  number  of  procedures.1 1  Thus,  the 
competency  levels  of  clinicians  performing  phlebotomy 
can  vary  widely. 

Safety  factors,  procedural  challenges,  and  lack  of  per¬ 
formance  standards  make  phlebotomy  a  particularly 
good  candidate  for  simulation-based  training.  To  date, 
however,  only  1  study  has  examined  VR  training  for 
phlebotomy.  Scerbo  et  al18  compared  the  CathSim  system 
with  simulated  limbs.  They  trained  third-year  medical 
students  for  up  to  2  hours  using  one  or  the  other  method 
and  assessed  performance  with  a  28-item  instrument.  In 
addition,  4  system-generated  performance  metrics  were 
recorded  for  the  students  who  trained  using  CathSim.  All 
the  participants  performed  a  pretest  and  posttest  on  the 
simulated  arm. 

Scerbo  et  al18  found  a  significant  improvement  between 
the  pretest  and  posttest  scores  only  for  the  students  who 
trained  with  the  simulated  arms.  Moreover,  the  posttest 
scores  of  the  simulated  arm  group  were  significantly  higher 
than  those  of  the  CathSim  group.  Furthermore,  although 
some  of  the  system-recorded  performance  metrics  showed 
a  significant  improvement  across  training  sessions,  the 
posttest  scores  did  not  reflect  the  benefits. 

Scerbo  et  al18  noted  that  the  2  training  systems  differed 
in  the  number  of  steps  that  could  be  addressed  from  all 
those  needed  to  perform  the  procedure.18  Consequently, 
the  investigators  reanalyzed  their  data  for  only  those  steps 
common  to  both  procedures.  This  second  analysis  showed 
a  pattern  of  results  similar  to  that  from  the  full  procedure. 


Posttest  scores  for  the  simulated-limb  group  were  statis¬ 
tically  higher  than  for  the  CathSim  group. 

To  gain  a  better  understanding  of  why  the  simulated 
limbs  resulted  in  better  performance,  Scerbo  et  al18  per¬ 
formed  a  physical  and  functional  analysis  of  each  simu¬ 
lator  system.  These  researchers  found  that  the  CathSim 
system  does  not  faithfully  represent  the  activities  required 
to  tie  a  tourniquet,  prepare  the  site,  stabilize  the  vein, 
and  orient  the  needle  properly  for  insertion.  In  addition, 
there  are  some  design-oriented  concerns.  For  instance,  the 
CathSim  trainer  is  designed  to  be  operated  by  a  seated  user 
interacting  with  the  computer,  whereas  phlebotomy  is 
typically  performed  with  the  phlebotomist  standing  and 
leaning  over  the  patient’s  arm.  Also,  because  the  CathSim 
system  requires  users  to  interact  simultaneously  with  both 
the  needle/hub  assembly  and  the  computer  mouse,  there 
is  greater  motor  interference  among  the  steps. 

Scerbo  et  al18  argued  that  the  physical  and  functional 
differences  between  the  2  simulator  systems  could  account 
for  the  observed  differences  in  performance.  They  also 
noted,  however,  that  because  both  groups  were  assessed 
on  the  simulated  limbs,  it  could  be  argued  that  the  students 
in  the  CathSim  group  were  at  a  disadvantage  because  they 
were  assessed  using  a  system  that  differed  from  the  one 
on  which  they  trained. 


•  EXPERIMENT-1 


One  goal  of  the  first  experiment  was  to  reexamine  the 
CathSim  and  simulated  limb  methods  for  phlebotomy 
training.  Toward  this  end,  participants  were  trained  using 
1  of  the  2  methods.  Again,  they  were  assessed  with  a  pretest 
and  posttest  on  the  simulated  limbs.  This  aspect  of  the 
study  was  retained  because  simulated  limbs  offer  a  safe 
and  standardized  platform  for  assessment.  Although  this 
approach  enables  comparisons  of  performance  before  and 
after  training,  a  clearer  comparison  of  the  2  methods  neces¬ 
sitates  that  performance  be  evaluated  with  a  “neutral” 
target  task.  Accordingly,  performance  was  assessed  using 
a  transfer  of  training  paradigm.  At  completion  of  the  post¬ 
test,  the  participants  were  assigned  to  a  clinic  and  assessed 
on  their  first  2  attempts  to  draw  blood  from  genuine 
patients.  This  additional  test  was  included  for  2  reasons. 
First,  if  the  differences  observed  by  Scerbo  et  al18  resulted 
from  assessing  CathSim  trainees  on  a  different  simulation 
system  (the  simulated  limbs),  then  a  subsequent  test  on  a 
neutral  platform  (ie,  genuine  patients)  would  eliminate 
the  unfair  disadvantage  for  the  CathSim  trainees.  Thus, 
all  trainees  would  be  assessed  on  a  platform  that  differed 
from  what  they  used  for  training. 

Second,  and  more  important,  because  the  ultimate  goal 
of  simulation-based  training  is  the  transfer  of  skills  to  the 
operational  environment,  the  current  paradigm  allows 
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healthcare  professionals  to  determine  the  relative  efficacy 
of  both  training  systems  for  performing  where  it  counts 
most — with  genuine  patients. 

Participants 

The  study  participants  were  45  third-year  medical  students 
at  Eastern  Virginia  Medical  School  who  participated  as 
part  of  their  family  clerkship  rotation  requirements.  The 
mean  age  of  participants  was  26.48  ±  3.23  years.  Some 
of  the  students  had  prior  medical  experience  (9  had  given 
intramuscular  injections,  7  had  given  subcutaneous  injec¬ 
tions).  Most  of  the  students  had  no  prior  experience  with 
medical  or  other  simulators.  However,  3  students  reported 
cursory  training  with  a  mannequin  arm  and  7  reported 
experience  in  a  driving  or  flight  simulator. 

Materials 

A  Life/Form  simulated  arm  (NASCO,  Inc.,  Fort  Atkinson, 
Wis.)  was  used  for  training  and  for  the  pretest  and  posttest 
assessments  (Figure  1).  The  simulated  arm  contains  latex 
veins  covered  by  vinyl  skin.  An  IV  bag  containing  artificial 
blood  is  hung  approximately  2  feet  above  the  arm  and 
connected  to  the  latex  veins  via  tubing.  Gravity  draws  the 
artificial  blood  into  the  arm’s  veins  and  simulates  bleeding 
when  the  skin  and  vein  are  punctured.  The  simulated  arm 
has  several  potential  sites  for  phlebotomy.  However,  only 
the  antecubital  fossa  was  used  in  this  study.  Other  mate¬ 
rials  also  required  for  phlebotomy  included  gloves,  tourni¬ 
quets,  alcohol  swabs,  gauze  pads,  20-gauge  needles,  and 
Vacutainer  (BD,  Franklin  Lakes,  NJ)  collection  tubes.  A 
biohazard  sharps  container  also  was  available  so  that  all 
needles  could  be  discarded  properly. 


The  CathSim  VR  system  was  used  in  this  study  (Fig¬ 
ure  1).  (Immersion  Medical,  Inc.  has  released  an  upgraded 
version  of  the  CathSim  system  since  completion  of  this 
study.)  This  system  has  a  phlebotomy  module  with  6  case 
patients.  Once  a  case  is  selected,  an  image  of  the  patient’s 
arm  appears  on  the  computer  screen,  and  the  system  guides 
the  participants  through  the  steps  of  selecting  an  insertion 
site,  palpating  the  site,  and  applying  the  tourniquet,  all 
via  the  computer’s  mouse.  Next,  a  magnified  image  of 
the  insertion  site  is  presented  on  the  screen  along  with  a 
virtual  needle.  The  mouse  is  used  to  position  the  virtual 
needle  for  insertion. 

After  choosing  the  insertion  site,  participants  apply 
traction  to  the  simulated  skin  pad,  fully  retract  the  needle 
unit  of  the  AccuTouch  device,  and  then  insert  it  into  the 
virtual  arm  while  monitoring  their  progress  on  the  screen. 
Once  the  needle  has  accessed  the  vein,  the  mouse  is  used 
to  click  on  icons  representing  the  collection  tubes.  Par¬ 
ticipants  attach  a  physical  tube  to  the  needle  unit  on  the 
AccuTouch  device.  If  vein  access  is  successful,  the  virtual 
collection  tube  on  the  screen  appears  to  fill  with  blood. 
Multiple  tubes  can  be  filled.  After  the  last  tube  has  been 
filled,  the  student  uses  the  mouse  to  release  the  tourniquet 
and  detach  the  tube  from  the  AccuTouch  device.  The 
procedure  is  completed  when  the  image  on  the  computer 
screen  displays  the  needle  being  deposited  into  a  biohazard 
sharps  container. 

Performance  Metrics 

Four  performance  metrics  from  the  CathSim  system  were 
used  to  evaluate  training  in  this  study.  The  first  was  a 
pain  factor  that  produces  a  score  from  1  (minimal  pain)  to 
10  (maximum  pain).  Pain  scores  increase  if  needle  pene¬ 
tration  is  too  deep  (ie,  the  needle  passes  through  both  vein 
walls),  the  needle  angle  is  inappropriate,  or  the  needle  is 
moved  after  it  is  embedded  in  the  skin.  The  second  metric 
was  hematoma  status.  Hematomas  occur  if  the  needle 


FIGURE  1. 

Trainee  performing  a  phlebotomy  on  the  simulated  arm  (left)  and  the  CathSim  system  (right). 
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scrapes  the  wall  of  the  vein  or  passes  through  both  vein 
walls.  The  third  metric  was  the  total  amountoft  time  the 
tourniquet  was  on  the  arm,  and  the  fourth  metric  was  the 
success  (ie,  if  the  student  successfully  completed  the  draw) 
or  failure  of  the  procedure. 

Assessment 

The  checklist  described  by  Scerbo  et  al18  was  used  in  this 
study  to  assess  pretest,  posttest,  and  field  test  perfor¬ 
mance.  The  checklist  addresses  23  steps  necessary  for 
performing  a  complete  phlebotomy  procedure  (eg,  site 
preparation,  needle  insertion  and  withdrawal)  and  3  steps 
concerning  overall  performance.  Checklist  scores  can 
range  from  0  to  88  points,  with  higher  scores  reflecting 
better  performance. 

Procedure 

The  participants  completed  a  background  questionnaire, 
including  demographic  information  and  prior  experience 
with  medical  procedures  and  simulators.  Then  15  partic¬ 
ipants  were  assigned  to  the  VR  (ie,  CathSim),  simulated 
arm  (SimArm),  and  clinic  group  conditions.  (Original  plans 
called  for  assigning  a  group  of  students  to  a  clinic  for  on- 
the-job  training  supervised  by  a  phlebotomist.  However, 
that  phlebotomist  left  employment  shortly  after  the  study 
began.  No  data  for  those  clinic-trained  students  could  be 
retrieved,  and  the  position  went  unfilled  for  much  of  the 
study.  Thus,  only  data  for  the  VR  and  SimArm  groups  are 
presented  in  this  article.)  All  the  participants  were  shown 
an  introductory  video  that  described  the  phlebotomy  pro¬ 
cedure.  After  the  video,  all  the  participants  performed  a 
phlebotomy  pretest  on  the  simulated  arm. 

All  the  participants  were  trained  individually.  The  VR 
group  was  trained  for  two  1-hour  sessions  separated  by 
1  week.  During  the  first  20  to  30  minutes  of  session  1, 
the  participants  were  introduced  to  the  CathSim  system 
and  instructed  on  how  to  use  the  AccuTouch  device.  The 
experimenters  assisted  them  with  their  first  procedure  and 
informed  them  about  the  performance  criteria  they  were 
required  to  meet.  For  the  remainder  of  the  first  session 
and  all  of  the  second  session,  the  participants  practiced  the 
procedure,  with  guidance  given  as  needed.  They  practiced 
on  all  6  case  patients  and  were  required  to  perform  a  suc¬ 
cessful  phlebotomy  on  each  patient.  Specifically,  they  prac¬ 
ticed  on  each  patient  until  they  met  the  following  criteria: 
a  pain  factor  of  3  or  less,  no  hematoma,  and  a  tourniquet 
time  less  than  90  seconds.  The  participants  had  to  meet 
these  criteria  before  they  could  progress  to  the  next  case 
patient.  The  6  patients  differed  slightly,  with  some  cases 
presenting  a  greater  challenge  than  others  (eg,  smaller 
veins,  less  visible  veins).  The  participants  began  training 


with  an  introductory  patient,  then  progressed  to  the  more 
■  -f  difficult  patients;  Also,  to  maximize  uniformity  across 
training  conditions,  the  participants  were  not  allowed  to 
use  the  side  or  transparent  views. 

Students  in  the  SimArm  group  practiced  phlebotomy 
on  the  arm  and  were  trained  in  accordance  with  the  items 
on  the  checklist.  During  the  first  10  minutes  of  the  first 
session,  they  were  reintroduced  to  the  equipment  they 
needed  for  the  phlebotomy  and  instructed  in  the  steps 
necessary  for  performance  of  the  complete  procedure: 
assemble  the  needle  unit,  tie  a  tourniquet,  palpate  the  vein, 
cleanse  the  site,  insert  the  needle,  depress  and  exchange  the 
Vacutainer  collection  tubes,  and  withdraw  the  needle. 
For  the  remainder  of  the  session,  the  participants  practiced 
the  procedure,  and  the  experimenter  provided  them  with 
verbal  feedback  regarding  their  performance.  The  first 
session  ended  when  the  student  was  able  to  perform  a 
phlebotomy  according  to  the  criteria  on  the  checklist  in 
2  consecutive  attempts. 

During  the  second  session,  the  students  were  once  again 
provided  with  verbal  feedback  regarding  their  performance 
until  they  could  meet  the  criteria  on  2  consecutive  attempts. 
On  average,  the  participants  in  this  group  completed 
their  first  session  in  30  to  40  minutes  and  their  second 
session  in  5  to  15  minutes. 

At  the  completion  of  training,  all  the  students  were  given 
a  posttest  on  the  simulated  arm,  and  their  performance  was 
assessed  against  the  checklist.  Next,  the  VR  and  simulated 
arm  participants  were  assigned  to  a  clinic  to  perform 
phlebotomy  on  actual  patients  under  the  supervision  of  a 
trained  phlebotomist  who  was  blind  to  the  group  assign¬ 
ments.  The  phlebotomist  assessed  the  participant’s  per¬ 
formance  on  the  first  2  phlebotomy  attempts  using  the 
same  checklist  used  for  the  pretest  and  posttest. 


Analyses  for  All  Participants 

An  overall  analysis  was  performed  on  the  data  from  the 
26  participants  who  completed  training  (12  in  the  VR 
group  and  14  in  the  SimArm  group).  A  2  method  x  2  test 
mixed  analysis  of  variance  (ANOVA)  was  conducted 
on  the  assessment  test  scores.  Test  (pretest,  posttest) 
was  the  within-participants  variable,  and  method  (VR  or 
SimArm)  was  the  between-groups  variable.  A  significant 
increase  in  assessment  scores  from  pretest  to  posttest  was 
found  for  all  the  participants  (F[l,24]  =  106.34;  P  <  .001; 
partial  f( 2  =  .82).  A  significant  interaction  also  was  found 
between  test  and  method  (F[l, 24]  =  51.95;  P  <  .001;  par¬ 
tial  fj2  =  .68). 

The  means  for  each  group  on  each  test  are  shown  in 
the  top  half  of  Table  1.  Tukey  HSD  post  hoc  analyses 
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TABLE  1 


Means  Assessment  Scores  and  Standard  Deviations  for  All  Participants 


VR 
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35^4.5  (7,0,9) 
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VR,  virtual  reality. 

demonstrated  that  the  SimArm  participants  scored  sig¬ 
nificantly  lower  than  the  VR  participants  on  the  pretest, 
putting  the  SimArm  participants  at  a  slight  disadvantage 
initially.  Both  groups  showed  a  significant  improvement 
from  pretest  to  posttest.  However,  the  posttest  scores  of 
the  SimArm  participants  were  significantly  higher  than 
the  posttest  scores  of  the  VR  participants. 

Analyses  for  Field  Test  Participants 

A  second  analysis  was  performed  on  the  data  from  only 
the  15  participants  from  the  VR  and  SimArm  groups 
who  completed  all  phases  of  the  study  including  the  field 
test.  In  the  field  test  phase,  the  participants  attempted  a 
phlebotomy  with  genuine  patients  and  were  assessed  by 
a  trained  phlebotomist.  The  participants  were  allowed 
to  perform  the  procedure  up  until  the  point  the  phle¬ 
botomist  thought  it  necessary  to  intervene  and  take  over. 
Thus,  field  test  assessment  scores  reflect  the  cumulative 
point  total  until  the  phlebotomist  had  to  intervene  (if 
necessary). 

It  should  be  noted  that  only  data  from  the  initial  attempt 
are  reported.  The  phlebotomist  who  supervised  the  par¬ 


ticipants  during  the  field  test  subsequently  revealed  that 
the  performance  of  many  participants  was  so  poor  that  it 
was  necessary  to  provide  remedial  training  before  allow¬ 
ing  them  to  make  another  attempt.  Consequently,  data 
from  the  second  attempt  are  contaminated  with  additional 
training  outside  of  the  experimental  protocol,  and  there¬ 
fore  are  not  presented. 

A  2  method  x  3  test  mixed  ANOVA  was  conducted, 
with  method  as  the  between-groups  variable  and  test  as 
the  within-participants  variable.  There  were  8  participants 
in  the  VR  group  and  8  in  the  SimArm  group.  A  significant 
main  effect  of  test  was  found  for  VR  and  SimArm  partic¬ 
ipants  (F[2,13]  =  10.09;  P  <  .01;  partial  f|2= .44).  The  main 
effect  of  test,  however,  was  moderated  by  a  significant 
interaction  with  method  (F[2,13]  =  4.66;  P  <  .05;  partial 
f)2  =  .26). 

Tukey  HSD  post  hoc  tests  demonstrated  that  the  field 
test  scores  for  VR  participants  were  significantly  lower 
than  their  pretest  or  posttest  scores,  which  did  not  differ 
from  one  another  (Table  2).  No  significant  differences  were 
found  among  the  SimArm  pretest,  posttest,  and  field  test 
scores.  However,  post  hoc  tests  did  show  that  the  SimArm 
participants  scored  significantly  higher  than  the  VR  par¬ 
ticipants  on  the  field  test. 


1  TABLE  2  J 

Means  and  Standard  Deviations  for  VR  and  SimArm  Participants  on 

All  Assessment  Phases 

idivvJ- 
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VR,  virtual  reality. 
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Analyses  of  Common  Steps 

As  noted  by  Scerbo  et  al,18  the  number  of  steps  from 
the  phlebotomy  procedure  that  can  be  trained  using  the 
CathSim  and  SimArm  methods  differ.  Thus,  an  assess¬ 
ment  of  the  14  steps  common  to  both  methods  described 
by  Scerbo  et  al18  also  was  conducted.  The  same  2  method 
x  2  test  mixed  ANOVA  also  was  performed  using  the  data 
from  the  26  participants  who  completed  training.  A  sig¬ 
nificant  increase  in  scores  from  pretest  to  posttest  on  the 
common  steps  was  found  for  all  the  participants  (F[l,24] 
=  81.86;  P  <  .001;  partial  fj2  =  .77).  A  significant  inter¬ 
action  also  was  found  between  test  scores  and  method 
(*[1,24]  =  30.84;  P  <  .001;  partial  t]2  =  .56). 

The  means  for  each  group  on  each  test  are  shown  in 
Table  1.  Tukey  HSD  post  hoc  analyses  of  the  means 
reflected  the  same  patterns  observed  for  the  full  procedure: 
the  SimArm  participants  scored  significantly  lower  than 
the  VR  participants  on  the  pretest.  Both  groups  showed 
a  significant  improvement  from  pretest  to  posttest.  How¬ 
ever,  the  posttest  scores  of  the  SimArm  participants  were 
significantly  higher  than  the  posttest  scores  of  the  VR 
participants. 

A  2  method  x  3  test  mixed  ANOVA  also  was  per¬ 
formed  using  the  data  from  the  15  participants  who  com¬ 
pleted  the  field  test.  A  significant  interaction  was  found 
between  test  and  method  (F[2,13]  =4.47;  P  <  .05;  partial 
f|2  =  .26).  A  Tukey  HSD  post  hoc  test  showed  that  the 
SimArm  participants  scored  significantly  higher  on  the 
field  test  than  the  VR  participants.  No  other  comparisons 
between  test  and  method  reached  significance. 

CathSim  METRICS 

The  performance  metrics  recorded  for  the  students  in  the 
CathSim  group  during  each  training  session  are  shown  in 
Table  3.  The  means  for  each  metric  were  determined  by 
collapsing  the  data  across  cases  and  trials  within  cases. 
Although  the  trends  for  all  the  metrics  suggest  improvement 
from  session  1  to  session  2,  only  the  decrease  in  hematomas 
reached  statistical  significance  (t[ll]  =  2.87;  P  <  .05). 


TABLE  3 


Performance  Metrics  for  the  CathSim  System  for  Each 
Training  Session  Collapsed  Across  Trials  and  Cases 

Metric  ,  Session  1  Session  2 

Mean  no.  of  unsuccessful  .  -  1.83  (2.82)  /  ,  -  0.67  (1.23) 
procedures  ' 'r 

Mean  no.  of  hematomas  "3.25  (2.14)  ‘  1.33  (1.67)* 

Mean  tourniquet  88.53  (35.34)  76.97  (9.72). 

time  (sec)  r  t  .. 

Mean  pain  score  :  1  -  4^15(3.16)  '3.19  (2.89) 

Standard  deviations  in  parentheses. 

*  P  <  .05. 


•  DISCUSSION  -  ‘ -r. , 

The  findings  from  the  current  study  show  an  advantage 
for  the  simulated  limbs  over  the  CathSim  system.  Although 
all  the  participants  improved  from  pretest  to  posttest 
irrespective  of  training  method,  the  posttest  scores  of  those 
in  the  SimArm  group  were  significantly  higher  than  those 
who  trained  with  the  CathSim  system.  Because  Scerbo 
et  al18  noted  that  each  training  method  addresses  a  dif¬ 
ferent  number  of  steps  than  the  complete  procedure,  a 
second  analysis  was  performed  on  only  those  steps  com¬ 
mon  to  both  methods.  The  findings  were  similar.  Although 
all  groups  showed  improvement  from  pretest  to  posttest, 
the  group  that  trained  with  the  simulated  limbs  achieved 
significantly  higher  posttest  scores  than  the  CathSim  group 
even  on  the  common  steps.  Collectively,  these  results  sup¬ 
port  the  earlier  findings  of  Scerbo  et  al,18  and  suggest 
that  the  physical  and  functional  differences  between  the 
2  simulator  systems  may  transcend  the  steps  common 
to  both  systems  and  account  for  the  observed  training 
differences. 

Although  the  findings  from  the  current  study  replicate 
those  of  Scerbo  et  al,18  it  still  could  be  argued  that  the 
participants  who  trained  with  the  CathSim  system  were 
assessed  on  a  target  task  that  differed  from  the  one  on 
which  they  practiced.  Thus,  a  second  goal  of  the  current 
study  was  to  use  a  transfer  of  training  approach  and  exam¬ 
ine  the  performance  of  both  groups  on  a  neutral  platform 
that  differed  from  their  training  environment.  Toward 
this  end,  all  the  participants  were  reexamined  on  their 
first  attempt  at  phlebotomy  with  genuine  patients. 

The  field  test  data  with  actual  patients  showed  2  impor¬ 
tant  findings.  First,  the  performance  gains  observed  on 
the  posttest  were  not  sustained  when  participants  had  to 
perform  an  actual  phlebotomy  on  a  genuine  patient.  Train¬ 
ing  with  either  method  did  not  transfer  well  to  the  oper¬ 
ational  environment. 

Of  course,  a  disadvantage  of  using  real  patients  to  eval¬ 
uate  training  methods  is  the  lack  of  methodologic  control 
over  patient  characteristics.  The  students  in  this  study  were 
asked  to  perform  a  phlebotomy  on  the  first  patient  who 
showed  up  at  the  clinic  during  their  session.  Although  the 
clinic  records  demographic  information  concerning  its 
patient  population,  these  data  were  not  collected  in  the 
current  study.  Thus,  although  it  is  possible  that  the  students 
trained  under  each  method  may  have  seen  patients  with 
more  (or  fewer)  challenging  characteristics,  one  would 
expect  individual  patient  characteristics  to  be  distributed 
randomly  across  a  group  assignment. 

Second,  although  all  the  students  had  difficulty  with 
genuine  patients,  there  was  a  clear  difference  between  the 
2  groups.  Specifically,  no  statistical  differences  were  found 
across  tests  for  the  SimArm  group.  Thus,  there  was  no 
training  advantage  or  disadvantage  for  these  students 
when  they  performed  their  first  phlebotomy  on  an  actual 
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patient.  By  contrast,  there  was  a  significant  drop  in  per¬ 
formance  on  the  field  test  for  those  who  trained  with  the 
CathSim  system  relative  to  their  pretest  and  posttest  scores. 
Furthermore,  their  field  test  scores  were  significantly  lower 
than  those  of  the  group  that  trained  with  the  simulated 
limbs.  A  similar  pattern  of  results  also  was  observed  when 
only  the  steps  common  to  both  methods  were  observed. 

Although  on  the  surface  these  results  show  that  there 
are  problems  with  the  CathSim  system  for  phlebotomy 
training,  it  is  important  to  note  that  efforts  were  made  to 
keep  the  training  conditions  consistent  to  maintain  exper¬ 
imental  rigor.  Thus,  all  the  participants  were  restricted  to 
no  more  than  2  hours  of  practice.  Unlike  the  simulated 
limbs,  the  CathSim  system  is  composed  of  several  com¬ 
puter  devices,  and  trainees  must  spend  a  fair  amount  of 
time  learning  how  to  coordinate  interactions  among  the 
components  to  perform  the  procedure.  For  some  of  the 
participants  in  the  CathSim  condition,  much  of  the  initial 
training  session  was  dedicated  to  learning  how  to  oper¬ 
ate  the  system.  Thus,  it  is  possible  that  participants  in  this 
condition  were  not  given  sufficient  time  both  to  learn  the 
simulator  system  and  to  practice  the  procedure  itself. 
Also,  although  the  CathSim  system  provides  2  auxiliary 
views  (side  and  transparent  views)  to  facilitate  training,  the 
participants  were  not  allowed  to  access  these  views.  Thus, 
the  participants  in  the  CathSim  condition  were  unable  to 
take  advantage  of  visual  aids  that  could  facilitate  training. 


•  EXPERIMENTS';;.;^:; 

■  i  -i.  ■. ;  sSiyJETj  Si 


A  second  study  was  performed  that  removed  the  training 
restrictions  imposed  in  experiment  1  to  maintain  consis¬ 
tency  across  conditions.  The  objective  in  experiment  2  was 
to  provide  trainees  with  an  extended  period  to  practice 
phlebotomy  on  the  CathSim  system.  We  expected  that 
performance  would  improve  as  a  result  of  continued  prac¬ 
tice.  Furthermore,  we  hypothesized  that  if  the  poorer  post¬ 
test  scores  of  those  trained  with  the  CathSim  system  in 
experiment  1  were  attributable  to  unfamiliarity  with  the 
simulator  system,  then  providing  additional  practice  time 
should  eliminate  that  disadvantage.  In  addition,  the  2  visual 
performance  aids  were  studied  to  determine  their  effect 
on  skill  acquisition.  We  expected  that  the  visual  aids  would 
have  a  positive  effect  on  the  rate  of  learning. 


their  participation.  Only  2  of  the  participants  reported 
prior  experience  in  the  medical  field,  with  1  indicating 
some  experience  with  phlebotomy  (ie,  drawing  blood  from 
animals).  No  other  students  reported  experience  with 
phlebotomy  or  giving  of  injections.  Seven  of  the  students 
did  report  experience  with  either  a  flight  or  driving  sim¬ 
ulator,  and  all  the  students  reported  computer  usage  of 
20  to  50  hours  (mean,  33.3  ±  10.31  hours)  per  week. 


Materials 

The  same  simulated  limbs  and  CathSim  system  used  in 
experiment  1  also  were  used  in  experiment  2. 


Procedure 

All  the  participants  watched  the  same  instructional  video 
about  phlebotomy  used  in  experiment  1,  then  performed 
a  pretest  on  the  simulated  arm.  Each  participant  then 
received  5  hours  of  individualized  training  over  2  sepa¬ 
rate  sessions  using  all  6  case  patients  provided  with  the 
CathSim  system.  All  the  participants  were  required  to  meet 
the  same  performance  criteria  established  in  experiment  1 
(ie,  a  pain  factor  of  3  or  less,  no  hematoma,  and  a  tourni¬ 
quet  time  less  than  90  seconds)  for  each  case  during  their 
training  sessions. 

Once  training  was  complete,  all  the  participants  per¬ 
formed  a  posttest  on  the  simulated  limb.  Half  of  the  par¬ 
ticipants,  assigned  at  random,  were  allowed  to  use  the 
2  visual  aids:  a  cutaway  side  view  that  showed  the  needle 
penetrating  the  arm  and  a  transparency  view  of  the  under¬ 
lying  vascular  system.  They  were  permitted  to  access  the 
visual  aids  as  often  as  they  wished.  The  remaining  par¬ 
ticipants  practiced  without  the  visual  aids. 

Performance  on  the  pretest  and  posttest  was  assessed 
with  the  same  28-item  checklist  used  in  experiment  1.  Skill 
acquisition  was  measured  by  the  time  taken  to  reach  the 
90-sec  criterion.  This  measure  was  chosen  because  3  cri¬ 
teria  (pain  factor,  presence  of  hematoma,  and  tourniquet 
time)  had  to  be  met  before  the  participant  could  move  on 
to  another  case. 


•  RESULTS 


•  METHODS 


Participants 

The  participants  in  experiment  2  were  10  graduate  students 
in  a  psychology  program  at  Old  Dominion  University.  They 
ranged  in  age  from  22  to  32  years  (mean,  25.8  ±  3.4  years). 
All  the  participants  received  monetary  compensation  for 


Pretest  and  Posttest 

One  individual  did  not  complete  the  training,  so  the  results 
are  based  on  9  participants.  Initial  analyses  showed  no 
effects  of  the  visual  aids.  Thus,  the  data  were  collapsed 
across  groups.  Pretest  and  posttest  scores  were  compared 
using  a  dependent  t  test.  Performance  improved  signifi¬ 
cantly  from  pretest  (mean  =  48,56,  SD  =  10.24)  to  posttest 
(mean  =  63.89,  SD  =  10.13;  r[8]  <.01). 
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Learning  Curve 

Each  participant  required  a  different  number  of  attempts 
to  reach  the  criterion  on  each  case  patient.  The  mean  total 
times  required  to  reach  the  criteria  for  all  the  cases  were 
computed  by  collapsing  the  data  across  attempts  and  par¬ 
ticipants.  The  resultant  means  are  shown  in  Figure  2.  As 
can  be  seen  in  the  figure,  most  of  the  performance  gain  was 
achieved  within  the  first  6  or  7  trials,  and  there  was  little 
additional  improvement  beyond  IS  cases.  The  abnormally 
long  time  for  case  18  was  attributable  to  a  single  student 
who  struggled  repeatedly  with  one  of  the  more  difficult 
cases.  A  line  of  best  fit  (y  =  5 10.92  x  -0.3  8 1 )  also  is  shown 
in  the  figure  with  case  18  excluded  from  that  data.  The  line 
represents  a  good  fit  with  the  data  ( R 2  =  .488;  P  <  .001). 


Comparison  Between  Experiments  1  and  2 

Another  way  to  examine  the  effects  of  extended  practice  on 
the  CathSim  system  is  to  compare  the  posttest  scores  from 
experiments  1  and  2.  The  participants  in  experiment  1 
received  up  to  2  hours  of  training  and  achieved  a  mean 
posttest  score  of  68.50  ±  8.02.  The  participants  in  exper¬ 
iment  2  received  5  hours  of  training  and  achieved  a  mean 
posttest  score  of  63.89  +  10.13.  The  results  of  an  inde¬ 
pendent  t  test  showed  that  the  means  did  not  differ  sig¬ 
nificantly  (*[19]  =-1.17;  P  >  .05). 


•  DISCUSSl£)fS 

••  to 


The  primary  goal  of  experiment  2  was  to  examine  the  role 
of  extended  training  on  the  CathSim  system.  Overall,  the 
participants  showed  improvement  between  the  pretest  and 
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FIGURE  2. 

Total  time  needed  to  reach  criterion  collapsed  across 
participants.  The  heavy  black  line  represents  the  line  of 
best  fit. 


posttest,  but  the  gains  were  not  dramatic.  The  posttest 
scores  were  only  30%  higher  than  the  pretest  scores  after 
5  hours  of  practice.  Moreover,  the  mean  posttest  scores 
from  experiment  2  were  equivalent  to  the  mean  posttest 
scores  of  the  CathSim  participants  observed  in  experi¬ 
ment  1.  Thus,  the  results  from  experiment  2  suggest  that 
giving  trainees  an  additional  3  hours  of  practice  beyond 
the  2  hours  provided  in  experiment  1  is  unlikely  to  pro¬ 
duce  any  appreciable  improvements  in  performance.  The 
results  regarding  criterion  time  showed  that  performance 
became  fairly  stable  after  only  a  few  cases  (-45  minutes), 
but  still  remained  variable  across  the  entire  training  ses¬ 
sion.  It  is  possible  that  much  of  this  variability  can  be 
attributed  to  the  different  levels  of  difficulty  associated 
with  each  unique  case  patient. 

Another  goal  of  experiment  2  was  to  examine  how 
visual  aids  facilitate  skill  acquisition  with  a  VR  simulator 
for  phlebotomy.  The  results  showed  no  appreciable  per¬ 
formance  benefit  from  providing  users  with  a  cutaway 
side  view  or  a  transparent  view  of  the  underlying  vascular 
system.  The  participants  in  both  groups  learned  at  the  same 
rate  and  scored  similarly  on  the  posttest. 

It  should  be  noted  that  the  participants  in  experiments 
1  and  2  were  drawn  from  different  populations.  Medical 
students  were  used  in  experiment  1,  but  a  decision  was 
made  to  use  graduate  students  in  experiment  2  because 
of  the  limited  number  of  medical  students  available  to 
participate  in  this  study.  We  believe  that  the  difference  in 
subject  populations  is  unlikely  to  account  for  the  perfor¬ 
mance  differences  between  the  2  experiments  because  the 
demographic  characteristics  of  the  medical  and  graduate 
students  were  similar,  and  because  both  sets  of  students  had 
little  or  no  prior  experience  with  phlebotomy.  Further¬ 
more,  because  the  focus  of  the  study  was  aimed  at  motor 
skill  performance,  knowledge  of  medicine  would  not  have 
been  an  advantage  for  the  medical  students. 

It  also  could  be  argued  that  student  nurses  would  have 
been  a  more  suitable  population.  Again,  participants 
who  were  naive  regarding  phlebotomy  were  recruited  so 
that  the  trainees  could  not  rely  on  their  previous  or 
related  experiences  to  accomplish  the  task.  In  this 
respect,  one  would  expect  all  participants,  regardless  of 
career  path,  to  be  equally  naive.  In  addition,  Scerbo  et  al9 
also  conducted  a  comparison  of  the  CathSim  system  and 
simulated  limbs  for  IV  cannulation.  The  participants  in 
their  study  were  students  in  a  physician  assistant  pro¬ 
gram.  Although  this  also  is  not  a  nursing  population,  it 
is  a  group  likely  to  perform  IV  cannulation  in  their  duties. 
Scerbo  et  al9  found  the  same  pattern  of  results  in  that  ear¬ 
lier  study  as  found  in  experiment  1  of  the  current  study. 
The  posttest  scores  of  those  trained  with  simulated  limbs 
were  significantly  higher  than  those  in  the  CathSim 
group.  Thus,  it  is  more  likely  that  the  pattern  of  results 
observed  in  each  study  is  attributable  to  differences 
between  the  2  training  methods  and  not  the  populations 
from  which  the  participants  were  drawn. 
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•  GENERAL  DISGUSSliS 

The  goal  of  the  reported  experiments  was  to  examine  the 
efficacy  of  the  CathSim  system  and  simulated  limbs  for 
phlebotomy  training.  In  a  previous  experiment,  Scerbo 
et  al18  found  that  simulated  arms  resulted  in  greater  per¬ 
formance  improvements  than  the  CathSim  system.  A  sim¬ 
ilar  pattern  of  results  was  observed  in  the  current  study. 
However,  students  trained  under  both  methods  in  the  cur¬ 
rent  study  were  assessed  for  their  ability  to  perform  phle¬ 
botomy  on  genuine  patients.  The  results  showed  that  those 
trained  on  the  CathSim  system  performed  more  poorly 
when  they  attempted  an  actual  procedure.  Furthermore, 
the  results  from  the  second  study  showed  that  extended 
training  on  the  CathSim  system  and  auxiliary  visual  aids 
are  unlikely  to  change  the  pattern  of  results  observed 
with  genuine  patients. 

Scerbo  et  al18  performed  a  functional  and  physical 
analysis  of  CathSim  and  simulated  limb  systems  and  argued 
that  2  important  differences  between  the  training  systems 
could  explain  the  poorer  performance  of  the  CathSim 
students.  First,  the  CathSim  system  allows  trainees  to 
practice  fewer  steps  than  the  full  procedure  includes.  Sec¬ 
ond,  the  manner  in  which  some  steps  are  implemented  in 
the  VR  system  differs  significantly  from  how  those  steps 
are  performed  in  an  actual  procedure.  For  example,  the 
CathSim  system  does  not  allow  the  user  to  choose  where 
to  place  the  tourniquet.  Moreover,  when  the  tourniquet 
is  placed,  it  appears  automatically  tied  in  the  proper  loca¬ 
tion.  Thus,  trainees  never  learn  how  to  place  and  tie  an 
actual  tourniquet.  This  particular  design  characteristic  may 
account,  in  part,  for  the  low  field  test  scores  observed  in 
the  first  study.  The  students  who  could  not  properly 
place  a  tourniquet  were  not  allowed  to  continue  with  the 
procedure. 

It  is  important  to  note  that  neither  system  completely 
or  faithfully  represents  the  phlebotomy  task  as  it  is  per¬ 
formed  in  practice.  Therefore,  it  should  not  be  surprising 
that  the  students  in  this  study  had  some  difficulty  trans¬ 
ferring  the  skills  they  acquired  during  training  to  an  actual 
procedure  with  a  genuine  patient.  For  example,  neither 
device  allows  trainees  to  practice  palpation  of  the  insertion 
site.  Furthermore,  the  choice  of  insertion  site  poses  no  real 
challenge  with  either  system.  On  the  simulated  arm,  the 
veins  protrude  from  the  arm,  making  site  selection  obvious. 
Also,  trainees  cannot  inspect  the  site  for  bleeding  after 
removing  the  needle,  as  would  be  done  with  a  genuine 
patient.  On  the  CathSim  system,  the  computer’s  mouse  is 
used  to  select  a  site  and  allows  palpation  in  only  specific 
areas  of  the  arm.  Finally,  neither  system  addresses  the  order 
in  which  collection  tubes  must  be  filled  to  minimize  carry¬ 
over  contamination  effects  from  additives  in  the  tubes. 15 

Despite  the  limitations  observed  with  the  CathSim 
system,  it  does  provide  certain  advantages.  One  of  the 
valuable  features  is  that  the  system  records  a  set  of  the 


objective  performance  metrics  that  are  not  available  or 
easily  obtainable  with  simulated  arms.  For  example,  the 
CathSim  system  records  an  index  of  hand  steadiness 
(ie,  reflected  in  the  pain  measure).  Also,  the  system  can 
generate  a  hematoma  on  the  basis  of  the  user’s  perfor¬ 
mance,  which  is  not  observable  with  the  simulated  limbs. 
In  fact,  in  the  current  study,  the  students  did  produce  sig¬ 
nificantly  fewer  hematomas  across  training  sessions. 

There  is  one  other  important  difference  between  sim¬ 
ulated  limbs  and  the  CathSim  system.  Training  with  sim¬ 
ulated  limbs  requires  the  use  of  expendable  supplies 
(eg,  needles,  collection  tubes,  alcohol  swabs).  Thus,  there 
are  costs  associated  with  simulated  limbs  every  time  the 
procedure  is  practiced.  All  “materials”  used  in  the  CathSim 
system  are  virtual,  thus  allowing  an  unlimited  number  of 
practice  trials  with  no  additional  costs. 

Overall,  the  CathSim  system  does  provide  some  critical 
measures  of  performance  that  would  be  observed  with 
actual  patients,  but  not  with  the  simulated  arms.  There 
are  advantages  and  disadvantages  to  both  the  CathSim 
system  and  the  simulated  arms  for  phlebotomy  training. 

Phlebotomy  requires  knowledge  of  blood  collection 
procedures  as  well  as  the  skills  needed  to  obtain  a  good 
sample  for  analysis.  Although  there  are  numerous  training 
courses  that  address  the  procedures,  standards  regarding 
the  performance  aspects  still  are  lacking. 

In  the  current  study,  2  simulator-based  methods  for 
training  phlebotomy  skills  were  examined.  The  results 
showed  a  clear  advantage  for  simulated  limbs  over  the 
VR-based  CathSim  system.  The  simulated  limbs  expose 
trainees  to  more  steps  of  the  procedure  and  allow  them 
to  practice  in  a  manner  more  consistent  with  how  the 
procedure  actually  is  performed.  These  advantages  became 
obvious  when  participants  attempted  to  perform  phle¬ 
botomy  with  a  genuine  patient.  Thus,  simulated  limbs 
are  likely  to  remain  the  method  of  choice  for  phlebotomy 
training. 

As  noted  earlier,  however,  each  simulator  system  does 
have  its  own  strengths  and  weaknesses,  and  neither  system 
faithfully  represents  all  the  steps  needed  to  perform  the 
procedure.  Thus,  the  optimal  training  program  may  be  one 
that  supplements  simulated  limbs  with  a  VR-based  system. 
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Abstract.  A  simulation-based  training  system  for  surgical  wound  debridement  was 
developed  and  comprises  a  multimedia  introduction,  a  surgical  simulator  (tutorial 
component),  and  an  assessment  component.  The  simulator  includes  two  PCs,  a 
haptic  device,  and  mirrored  display.  Debridement  is  performed  on  a  virtual  leg 
model  with  a  shallow  laceration  wound  superimposed.  Trainees  are  instructed  to 
remove  debris  with  forceps,  scrub  with  a  brush,  and  rinse  with  saline  solution  to 
maintain  sterility.  Research  and  development  issues  currently  under  investigation 
include  tissue  deformation  models  using  mass-spring  system  and  finite  element 
methods;  tissue  cutting  using  a  high-resolution  volumetric  mesh  and  dynamic 
topology;  and  accurate  collision  detection,  cutting,  and  sofl-body  haptic  rendering 
for  two  devices  within  the  same  haptic  space. 

Keywords.  Surgical  debridement,  wound  debridement,  training,  virtual  agent, 
multimedia,  debris  modeling,  prototype,  augmented  reality. 

Introduction 

In  1999,  The  Institute  of  Medicine  issued  a  report  estimating  that  as  many  as  98,000 
people  die  from  errors  in  hospital  settings  each  year  [1],  Subsequently,  the  AMA’s 
Accreditation  Council  for  Graduate  Medical  Education  limited  the  time  that  residents 
can  be  required  to  work  to  80  hours  a  week.  Although  this  action  was  intended  to 
improve  patient  safety,  it  also  reduced  the  number  of  patient-contact  hours  needed  to 
train  residents.  Thus,  medical  educators  are  looking  for  alternative  methods  to  provide 
residents  with  meaningful  learning  experiences.  Many  are  turning  to  simulator-based 
training  to  meet  that  need  [2,  3], 

In  many  high  risk  domains  (e.g.,  aviation,  military  training,  nuclear  process  plant 
control,  etc.),  simulators  are  often  used  to  allow  trainees  to  acquire  and  practice  skills 
in  a  safe  and  controlled  environment  [4],  One  of  the  important  benefits  of  this  approach 
is  that  trainees  can  acquire  much  of  the  fundamental  knowledge  and  skills  needed  to 
perform  elementary  activities  largely  on  their  own  without  the  need  for  constant 
supervision  by  an  instructor.  Accordingly,  that  approach  was  adopted  as  the  guiding 
principle  behind  the  development  of  the  surgical  wound  debridement  simulator  system. 

I.  Wound  Debridement 

Wound  debridement  refers  to  the  process  of  removing  necrotic,  devitalized,  or 
contaminated  tissue  and/or  foreign  material  to  promote  healing  [5].  In  surgical 


debridement,  a  scalpel  or  scissors  is  used  to  cut  away  the  necrotic  tissue.  Wound 
debridement  is  a  minor  surgical  procedure,  but  one  that  is  performed  by  surgeons, 
physicians  assistants,  surgical  assistants,  nurses,  and  military  medics  and  corpsmen.  At 
present,  this  procedure  is  typically  learned  with  actual  patients  under  the  guidance  of  a 
more  experienced  physician. 

2.  The  Simulation-Based  Training  System 

A  simulation-based  virtual  reality  (VR)  training  system  for  surgical  wound 
debridement  was  developed  to  provide  trainees  with  the  fundamental  knowledge  and 
skills  to  perform  the  procedure  through  self-instruction  and  practice,  thereby 
eliminating  the  need  for  one-on-one  instruction.  The  system  comprises  a  multimedia 
introduction,  a  surgical  simulator  (tutorial  component),  and  an  assessment  component. 

2. 1  Multimedia  Training  Component 

The  multimedia  training  component  describes  varieties  of  wound  categories  (e.g., 
burns,  lacerations,  etc.);  methods  of  debridement  (e.g.,  sharp,  mechanical,  etc.),  and 
equipment/materials  used  in  the  procedure.  This  module  provides  the  pedagogical 
context  for  skills  training  and  assessment,  linking  together  all  of  the  required  elements 
for  mastering  the  procedure.  Pedagogical  material  and  methodologies  have  been 
researched  (e.g.,  [5])  and  the  system  design  stems  from  training  requirements  and 
iterative  development  obtained  through  consultation  with  domain  experts. 

Modules  are  arranged  in  sequential  order  to  support  educational  advancement  of 
the  novice;  however,  the  navigational  menu  design  permits  all  items  to  be  accessible  at 
any  time  thereby  also  supporting  the  experienced  user.  Further,  freedom  to  run  the 
multimedia  component  on  a  platform  other  than  the  one  used  for  the  skills  training 
makes  it  possible  to  support  multiple  users  simultaneously. 

2.2  Virtual  Agent 

A  software  virtual  agent  is  included  and  performs  the  role  of  an  instructor  by  providing 
verbal  guidance  and  assessment  feedback  (see  Figure  la).  Designed  to  be 
comprehensive  in  order  to  reduce  the  need  for  a  more  senior  instructor,  the  software 
addresses  the  patient’s  condition,  initial  description  of  the  injury,  and  provides 
instruction  on  operation  of  the  simulator.  Specific  instructions  cover  the  removal  of 
foreign  objects  with  forceps,  scrubbing  with  a  brush,  rinsing  with  saline  solution,  and 
the  maintenance  of  sterility.  The  trainee  is  prompted  to  move  on  when  the  current  stage 
is  satisfactorily  completed.  The  current  software  implementation  for  the  agent  uses 
Flaptek®1  and  NeoSpeech2. 

2.3  The  Simulator:  General  Description  and  Technical  Implementation 

A  prototype  of  the  wound  debridement  trainer  has  been  developed  using  the  Derivative, 
Inc.,  API3,  for  fast  initial  testing  of  user  and  design  requirements  (see  Figure  lb). 


1  Haptek®,  PeoplePutty,  Haptek  Player  SDK,  Haptek®  Player  viewed  10-17-05,  http://www.haptek.com 

2  NeoSpeech,  Kate,  viewed  10-17-05,  http://www.nextup.com/neospeech.html 

3  Derivative  Inc.  Touch  Designer,  viewed  10-17-05.  http://www.derivativeinc.com/ 


OpenHaptics™  Toolkit  (SensAble,  Inc.)  interfaces  with  a  Phantom®  haptic  device 
(Omni™  or  Premium)  for  wound  interaction  while  viewing  it  on  a  Reachin  display 
device  4,s.  A  parallel  research  effort  with  an  open  architecture  uses  OpenSceneGraph 
alongside  an  FEM  physics  approach  for  increased  flexibility  and  realism  in  the  training 
application  (see  Section  3.2). 


Figure  la,  b.  Simulator  system  for  surgical  wound  debridement:  a)  virtual  agent  tutor  shown  on  the  left  and 
b)  prototype  system  for  removing  glass  shards  from  thigh  laceration  shown  on  the  right. 

2.3. 1  The  Anatomy  and  Wound 

The  simulator  provides  training  on  a  shallow  laceration  wound  to  the  thigh  caused  by  a 
motorcycle  accident.  Our  wound  research  focuses  on  the  extremities  and  the  leg  model 
is  derived  from  the  National  Library  of  Medicine’s  Visible  Human  Project  [6].  Axial 
anatomical  imagery  is  manually  segmented,  registered,  and  triangulated  polygonal 
surface  meshes  of  relevant  objects  are  constructed.  Subsequent  to  viewport  culling,  the 
skin  mesh,  as  used  in  our  prototype,  is  approximately  1000  polygons. 

Foreign  material  embedded  in  the  wound  is  modelled  both  geometrically  and  as 
surface  texture.  It  is  possible  to  render  a  high  degree  of  dirt  without  an  increase  in 
computational  expense  as  would  be  the  case  if  debris  implementation  were  pursued 
solely  as  geometry.  Currently  the  majority  of  the  debris  (dust,  dirt)  is  rendered  as 
texture  on  the  wound  surface  geometry  while  larger  objects  are  polygonal  meshes. 

The  simulation  uses  several  textures  for  a  range  of  the  debridement  processes  (see 
Figure  2).  Each  polygon  in  the  mesh  is  handled  separately  in  terms  of  texture  image 
while  texture  coordinates  remain  constant.  The  intersection  between  an  instrument  and 
surface  polygon  results  in  a  substitute  texture  being  called  for  that  polygon. 

Bleeding  is  implemented  using  a  particle  system.  Particles  are  created  only  when 
instruments  intersect  with  polygons  within  the  wound  area.  Small  amounts  of  gravity 
and  randomness  are  added  to  particle  motion.  Upon  collision  with  the  wound  surface,  a 
particle’s  reflecting  vector  is  reduced  normal  to  the  collision.  These  implementations 
contribute  to  a  realistic  rendering  where  particles  slow  down  and  approach  the  wound 
surface  geometry  over  the  duration  of  their  lifespan. 

4  SensAble  Technologies  OpenHaptics™  Toolkit,  Phantom®  Premium,,  Omni™,  viewed  10-17-05, 
hMp://www.sensable.com 

5  Reachin  Technologies  Reachin  Display,  viewed  10-17-05,  http://www.reachin.se/ 
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Figure  2  The  concept  of  using  textures  to  model  debris  around  the  wound. 


2.3.2  Graphics  and  Haptics  Simulation  within  the  Prototype  Application 


A  mass-spring  simulation  implemented  within  the  prototype  application  and  Derivative 
API  drives  graphics  deformation.  Approximately  200  nodes  are  deformed  to  restrict 
the  simulation  to  those  that  fall  inside  the  wound  area  (the  remainder  are  fixed).  All 
nodes  have  the  same  mass  and  drag  and  the  spring  is  kept  fairly  stiff  for  stability. 

A  haptic  model  that  does  not  update  topological  changes  from  the  deforming  mass 
spring  model  in  Derivative  has  been  implemented.  Users  do  not  readily  perceive  the 
limited  synchronicity  between  the  graphics  and  haptics  implying  an  effective  and 
economic  implementation.  The  haptic  rendering  also  supports  grabbing  points  on  both 
the  deforming  graphic  and  the  haptic  meshes  (e.g.,  with  forceps,  see  Figure  lb).  Force 
is  subsequently  applied  and  the  user  feels  tension  through  the  haptic  device.  Releasing 
the  tissue  updates  the  graphical  points  back  to  their  starting  position.  Both  graphics  and 
haptics  applications  of  the  simulator  will  run  on  a  single  laptop  computer  (i.e.,  with  3.2 
GHz  CPU;  1GB  RAM;  GeForceFX  Go  5700).  Running  in  parallel,  the  applications 
average  60Hz  for  graphics  and  300Hz  for  haptics. 


3.  Ongoing  Research  and  Development 

3. 1  Assessment 


At  present,  assessment  is  limited  to  the  amount  of  foreign  material  successfully 
removed  from  the  wound  and  the  time  needed  to  perform  the  procedure.  The  next 
generation  will  include  assessment  modules  for  the  didactic  content  addressing 
preoperative  and  postoperative  care,  performance-based  feedback  during  the  tutorial, 
summative  feedback  at  the  end  of  the  tutorial,  and  assessment  information  for  either  the 
trainee  or  instructor.  Performance-based  assessment  will  target  two  types  of  errors.  One 
set  of  errors  is  intended  to  be  “recoverable”.  For  example,  removal  of  a  very  large 
shard  of  glass  from  the  wound  could  cause  a  significant  amount  of  bleeding.  It  would 
then  be  necessary  for  the  trainee  to  stop  the  bleeding  and  proceed  with  the  debridement. 
The  second  type  of  error  is  intended  to  be  “unrecoverable”.  For  instance,  the  trainee 
might  damage  a  major  blood  vessel  or  nerve.  At  this  point  the  simulation  would 
terminate  and  feedback  would  be  provided  that  describes  the  seriousness  of  the  error, 
why  it  occurred,  and  how  to  avoid  it  in  the  future. 


3.2  Development  Platform 

There  are  many  fundamental  challenges  to  developing  a  simulation  system  for  an  open 
procedure  like  surgical  debridement.  To  address  the  issues  in  the  current  prototype 
using  the  Derivative  API,  a  new  solution  employing  more  capable  and  flexible 
technologies  is  currently  under  research  and  development.  The  following  briefly 
describes  several  major  components  under  research  and  development: 

■  Object  oriented  software  design  and  development  are  employed  to  ensure  software 
quality,  ease  of  component  integration,  future  extension,  and  maintenance.  Major 
classes  of  the  simulator  include  tissue  and  instrument  supporting  the  simulated 
objects  and  mass  spring  and  finite  element  models  underlying  the  physics-based 
models.  Some  auxiliary  classes  are  also  developed  to  facilitate  computation,  such 
as  topology  processing.  In  addition,  the  software  components  are  not  tied  to  a 
specific  operating  system  specific  and  facilitate  portability  to  other  platforms. 

■  OpenSceneGraph,  an  open  source  high  performance  3D  graphics  toolkit,  is  utilized 
for  developing  the  visualization  component  of  the  debridement  simulator.  Utilizing 
scene  graph  technology  and  the  latest  OpenGL®  2.0  graphics,  OpenSceneGraph 
has  strength  in  performance,  productivity,  portability,  and  scalability.  A  variety  of 
models  are  utilized  for  different  computations.  Volume  mesh  is  used  by  tissue 
deformation  and  cutting  simulation,  while  surface  mesh  used  by  irrigation 
simulation  and  collision  detection.  For  computational  efficiency,  instruments  are 
represented  by  a  line  segment(s)  enclosing  them  for  collision  detection  purposes. 

■  Removing  necrotic  or  contaminated  tissue  with  a  scalpel  or  scissors  is  a  critical 
procedure  that  differentiates  between  surgical  debridement  and  other  debridement 
approaches.  Thus,  simulation  of  tissue  cutting  in  a  surgical  debridement  simulator 
is  essential.  To  effectively  model  and  simulate  the  cutting  procedure,  a  volume 
mesh  is  utilized  to  represent  the  wound  region  on  the  thigh.  A  surface  mesh  is 
extracted  from  the  volume  mesh  to  render  the  thigh  graphically.  Tissue  cutting  is 
simulated  by  deleting  volume  elements  in  contact  with  the  cutting  instruments, 
with  color  images  from  the  visible  human  data  set  rendered  as  3D  textures. 

■  Mass-spring  system  and  finite  element  method  (FEM)  approaches  are  implemented 
to  simulate  soft  tissue  deformation.  Research  is  underway  to  build  fast 
computational  schemes  to  improve  real-time  performance  for  fast  FEM 
computational  methods,  rapid  neighborhood  construction,  and  a  hybrid  structure  of 
Mass-spring  and  FEM. 

■  XML,  a  simple  and  very  flexible  markup  language,  is  selected  as  the  file  format  to 
store  the  graphical  and  physical  properties  of  the  thigh  model,  including  geometry, 
texture  maps,  topology,  and  FEM  parameters.  XML  writer  and  reader  programs 
have  been  developed  to  create  and  parse  XML  files. 

The  finite  element  code  developed  for  the  wound  debridement  application  computes 
the  3D  volumetric  deformation  of  the  leg  model  based  on  the  displacement  of  a  few 
surface  nodes  as  determined  by  the  collision  detection  module.  The  current  model 
assumes  all  materials  are  either  linear  elastic  or  rigid.  Rigid  materials,  (e.g.,  bones), 
can  be  represented  by  applying  O-displacement  boundary  conditions  to  an  appropriate 
subset  of  the  model's  nodes.  The  stiffness  of  elastic  tissues  in  the  model  can  be  varied 
according  to  segmented  tissue  type. 

The  finite  element  code  was  written  specifically  for  the  purpose  of  interactive 
simulation.  It  makes  use  of  sparse  matrix  data  structures  and  efficient  LASPACK 


sparse  matrix  solution  routines.  Incremental  updates  to  the  stiffness  matrix  are  made  as 
the  model  mesh  is  cut  during  the  simulation.  The  finite  element  code  provides  as 
output:  (1)  the  displacement  of  every  node  in  the  model,  and  (2)  the  force  vector  in 
effect  at  every  node.  The  force  vector  that  is  computed  at  the  point  of  contact  between 
a  surgical  instrument  and  the  deformable  model  defines  the  force  that  should  be 
haptically  displayed  to  the  user.  Currently,  the  finite  element  code  computes  a 
deformation  solution  at  a  rate  of  60-70  Hz  for  a  model  with  400  elements  and  20-30  Hz 
for  a  model  with  1100  elements  which  is  sufficient  for  graphics.  Further  optimization 
of  the  code  to  improve  efficiency  and  a  parallel  implementation  is  planned. 

4.  Conclusion 

A  simulator-based  training  system  was  developed  for  surgical  wound  debridement.  The 
goal  was  to  create  a  system  that  would  allow  health  care  providers  to  acquire  the 
fundamental  knowledge  and  skills  to  perform  the  procedure  through  self-instruction. 
The  current  system  contains  a  multimedia  component  for  didactic  material,  a  tutorial, 
and  a  VR-based  simulator  with  haptic  feedback.  Trainees  can  learn  to  clean  to  a  wound 
and  remove  foreign  objects.  However,  many  technical  challenges  remain  to  improve 
the  realism  of  the  graphical  and  haptic  displays.  These  include:  1)  more  realistic  tissue 
deformation  using  mass-spring  and  finite  element  models,  2)  better  collision  detection, 
3)  incorporating  tissue  cutting,  and  4)  adding  two-handed  haptics  because  debridement 
is  typically  performed  using  both  hands.  It  is  expected  that  solutions  to  these  challenges 
will  not  only  help  to  refine  the  physics-based  models  underlying  the  anatomy  and 
physiology  represented  in  the  system,  but  will  also  expand  the  variety  of  wounds  that 
can  be  included  in  the  set  of  training  cases. 
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Abstract 

The  number  of  fatalities  attributable  to  medical  errors  in  the  United  States  has  recently  come 
under  intense  scrutiny.  In  response  to  this  problem,  medical  schools  are  seeking  alternative 
methods  to  train  their  students.  One  of  the  methods  currently  being  used  to  enhance  medical 
training  is  simulation.  In  surgery,  the  most  common  type  of  simulators  target  laparscopic 
procedures  and  the  Minimally  Invasive  Surgical  Trainer  Virtual  Reality  (MIST  VR)  is  the  most 
widely  studied  laparoscopic  simulator.  Despite  its  research  exposure,  however,  few  studies  have 
demonstrated  the  transfer  of  skills  learned  on  the  MIST  VR  to  procedures  performed  on  genuine 
patients.  Further,  most  of  the  research  to  date  is  not  tied  to  any  theories  of  learning.  The  purpose 
of  the  present  paper  is  to  describe  current  research  on  laparoscopic  training  with  simulators  and 
discuss  the  need  for  theories  of  learning  and  skill  acquistion  in  future  research. 
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Medically-Related  Deaths  and  the  Classical  Model 

Recent  research  reveals  that  patients  incur  an  increased  risk  to  their  safety  when  they 
receive  hospital  care.  In  a  report  issued  by  the  Institute  of  Medicine  (IOM),  it  was  estimated  that 
between  44,000  and  98,000  deaths  can  be  attributed  to  medical  errors  each  year  in  U.S.  hospitals 
(Kohn,  Corrigan,  &  Donaldson  1999).  Further,  it  was  noted  that  patient  care  associated  with 
surgery,  accounted  for  35-48%  of  all  adverse  events.  Statistics  such  as  these  have  helped  to 
sharpen  the  focus  on  patient  safety.  However  the  estimate  of  deaths  has  encouraged  other 
researchers  to  take  a  closer  look  at  the  context  in  which  these  deaths  occur.  Hayward  and  Hofer 
(2001)  offered  a  more  conservative  estimate  of  deaths  due  to  medical  error.  In  their  study,  they 
assessed  not  only  deaths  due  to  medical  error,  but  also  the  likelihood  that  the  death  could  have 
been  prevented  with  better  care.  Their  assessment  sampled  deaths  for  a  one  year  period  in  seven 
Veterans  Affairs  in  Virginia  hospitals.  ,  The  reviewers  determined  that  out  of  1 1 1  fatalities, 
death  might  have  been  prevented  in  23%  of  cases,  and  was  probably  preventable  in  6%  of  cases. 
The  authors  noted  that  these  percentages  are  similar  to  those  reported  in  other  studies  (e.g., 
Brennan,  Leape,  &  Laird  1991;  Hayward,  McMahon,  &  Bernard  1993)  which  reviewed  death 
rates  due  to  medical  error.  More  important,  however,  the  reviewers  judged  that  about  one  third 
of  the  patients  would  have  survived  for  only  another  three  months  or  so  had  they  been  given 
optimal  care.  These  findings  suggest,  that  the  estimate  of  hospital-related  death  rates  in  the  IOM 
report  due  to  medical  errors  may  have  been  inflated  because  the  population  upon  which  they 
were  based  is  often  older,  terminally  ill,  and  may  also  include  do  not  resuscitate  orders. 

Other  research,  however,  using  different  a  methodology  and  definitions  of  errors,  found 
that  the  earlier  estimate  of  deaths  reported  by  the  IOM  may  have  been  too  conservative.  The 
HealthGrades  report  (2004)  included  Medicare  data  from  all  50  states  over  a  three-year  period. 
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Patient  safety  incidents  (e.g.,  failure  to  rescue,  foreign  bodies  left  during  a  procedure,  infections 
due  to  medical  care,  etc.)  were  tallied  for  living  and  deceased  individuals.  The  findings  from  the 
HealthGrades  report  suggested  that  nearly  195,000  deaths  occur  each  year  because  of  hospital 
related  medical  errors.  Further,  if  patient  death  due  to  medical  error  was  listed  as  a  formal  cause 
of  death  by  the  Centers  for  Disease  Control  and  Prevention,  it  would  rank  number  six  among  all 
leading  causes.  More  important,  the  study  also  indicated  that  deaths  due  to  medical  error  have 
shown  no  signs  of  decreasing  from  the  rates  first  published  in  the  IOM  report. 

Errors  that  compromise  patient  safety  may  be  due,  in  part,  to  the  classic  teaching  model 
in  medicine.  Training  in  the  medical  profession  has  typically  followed  an  apprenticeship  model, 
Whereby  students  learn  and  train  under  the  supervision  of  a  more  experienced  professional 
(Kauffman,  2001).  Unfortunately,  this  model  may  create  inconsistent  training  and  practice 
across  students.  Shimada  et  al.  (1998)  expressed  several  disadvantages  associated  with  clinical 
training  under  the  apprenticeship  model:  a)  students  typically  will  not  encounter  all  possible 
pathologies  during  their  time  of  training,  which  can  hinder  their  ability  to  recognize  and  diagnose 
pathology  they  have  never  encountered  before;  b)  there  may  be  fewer  opportunities  for  less 
experienced  students  to  practice  uncommon  procedures  when  confronted  with  very  sick  patients; 
and  c)  there  is  a  need  for  patient  cooperation,  and  many  patients  may  choose  not  to  be  cared  for 
by  student  physicians. 

Satava  (2004)  noted  that  surgical  training  has  never  faced  a  more  challenging  set  of 
circumstances.  Financial  policies  concerning  reimbursement  continue  to  draw  down  funds 
needed  for  training.  Resident  work  hours  have  also  been  reduced  in  an  attempt  to  standardize 
training.  Others  have  also  noted  that  reductions  in  training  time  as  well  as  limited  operating 
room  exposure  can  produce  students  and  residents  who  have  deficient  technical  abilities 


Medical  Simulators  5 


(Feldman,  Sherman,  &  Fried,  2004).  Kauffman  (2001)  also  noted  two  other  sources  of  problems 
in  surgical  education:  1)  teachers  have  fewer  hours  to  train  residents,  and  2)  residents  have  fewer 
hours  of  patient  contact.  Twenty  years  ago,  patients  were  typically  admitted  to  the  hospital  for  a 
period  of  one  to  two  days,  which  allowed  residents  to  spend  many  hours  learning  from  each 
patient.  Today,  however,  patients  spend  substantially  less  time  in  the  hospital,  which  leads  to  a 
decrease  in  patient  contact  time  for  the  resident.  Fewer  hours  of  patient  contact,  in  turn,  may 
ultimately  result  in  missed  learning  opportunities  for  the  resident. 

Another  problem  with  the  reduction  in  resident  hours  concerns  the  opportunity  to  become 
technically  competent  on  a  wide  range  of  procedures.  Surgeons  are  expected  to  learn  and  master 
many  different  psychomotor  skills  despite  the  potential  inconsistencies  associated  with  their 
training,  such  as  exposure  to  different  types  of  surgery,  patients,  and  different  curriculum 
criteria.  Hamdorf  and  Hall  (2000)  noted  that  surgical  skills  require  three  general  stages  for 
mastery.  The  first  stage  is  associated  with  cognition  and  perceptual  awareness;  that  is,  an 
individual  must  understand  the  task  and  skills  to  be  performed  as  well  as  the  spatial  relations  and 
mechanical  requirements  involved.  During  the  second  stage,  the  newly  acquired  motor 
techniques  are  integrated  with  their  application.  In  the  third  stage,  the  techniques  are  practiced, 
thereby  permitting  the  development  of  new  motor  skills  that  can  be  performed  more  efficiently 
and  with  greater  precision.  Although  Hamdorf  and  Hall  were  describing  skill  acquisition  as  it 
relates  to  surgery,  an  extensive  literature  on  the  nature  of  psychomotor  skills  in  general  does 
exist. 

Skill  Acquisition 

Skills  can  be  described  as  an  individual’s  learned  capability  to  perform  specific  acts 
(Farmer  et  al.,  1999).  Skills  are  acquired  by  making  a  permanent  change  in  the  way  an  act  is 
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performed  through  intentional  or  incidental  learning.  Psychomotor  skills  are  coordinated 
muscular  movements  of  the  body  that  follow  a  motor  schema.  A  motor  schema  has  been 
described  as  a  memory  for  motor  skills  that  have  been  well  learned  and  operate  independently  of 
perceptual  feedback  information  (Schmidt,  1975).  Psychomotor  skills  can  be  divided  into  fine 
and  gross  categories  (Holding,  1987).  Fine  psychomotor  skills  are  those  that  usually  involve 
detailed  manipulations  of  a  specific  part  of  the  body  (e.g.,  fingers,  hands,  etc.).  Gross  motor 
skills  are  those  that  involve  whole-body  movements. 

In  order  for  an  individual  to  successfully  acquire  fine  psychomotor  skills  they  must 
practice.  One  of  the  most  important  issues  concerns  the  amount  of  practice  needed  to  learn  a 
psychomotor  skill.  Crossman  (1959)  demonstrated  that  motor  skill  acquisition  can  be  described 
with  respect  to  performance  speed  and  amount  of  practice.  Performance  speed  and  practice 
follow  a  log  power  function,  such  that  initially,  performance  speed  is  quite  slow  when  practicing 
the  task.  However,  with  more  practice,  performance  speed  increases.  The  increase  in 
performance  speed  is  thought  to  result  from  decreases  in  attentional  resource  demands  that  allow 
the  skill  to  be  performed  in  an  automated  fashion  (Wickens  &  Hollands,  2000).  However, 
performance  speed  does  not  increase  linearly  with  extended  practice.  Instead,  it  increases  at  a 
decreasing  rate  and  eventually,  additional  practice  on  the  task  will  result  in  only  minimal  gains  in 
performance  speed. 

Overlearning  is  another  issue  related  to  practice.  Wickens  and  Hollands  (2000)  noted 
that  overlearning  occurs  when  an  individual  continues  to  practice  after  he  or  she  has  met  the 
performance  criterion.  As  noted  previously,  overlearning  does  not  necessarily  decrease  the 
amount  of  time  it  takes  to  perform  the  task.  However,  overlearning  does  enable  the  individual  to 
make  the  newly  acquired  skill  more  automatic.  That  is,  the  motor  program  developed  through 
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additional  practice  comes  to  require  fewer  cognitive  resources.  Wickens  and  Hollands  (2000) 
suggested  that  for  skills  not  practiced  on  a  routine  basis,  overlearning  will  enhance  the 
probability  that  the  skills  will  be  retained  over  time. 

Another  important  aspect  of  practice  concerns  how  sessions  are  structured.  Specifically, 
schedules  of  practice  can  be  either  massed  (performed  all  at  one  time)  or  distributed  (performed 
in  intervals  over  a  period  of  time).  Farmer  et  al.  (1999)  discussed  the  advantages  and 
disadvantages  associated  with  each  schedule  of  practice.  For  psycho  mo  tor  skill  acquisition, 
training  sessions  that  are  massed  rather  than  distributed,  usually  result  in  poorer  skill  acquisition 
(Schmidt  &  Lee,  2005).  Also,  performance  assessed  immediately  after  completion  of  training  is 
poorer  and  more  variable  under  massed  practice  (Lee  &  Genovese,  1988).  Farmer  et  al.  (1999) 
noted  that  the  poorer  performance  associated  with  massed  practice  sessions  is  usually  due  to 
fatigue  as  well  as  decreases  in  motivation.  However,  if  distributed  practice  sessions  are 
employed,  performance  during  and  immediately  following  training  is  more  consistent.  When 
performance  is  assessed  for  skill  retention  both  massed  and  distributed  schedules  of  practice  can 
produce  both  similar  and  dissimilar  effects  (Farmer  et  al.  1999;  Holding,  1987).  Discrepancies  in 
the  literature  may  be  due  to  fatigue,  retention  intervals,  durations  of  work/rest  cycles,  and  the 
nature  of  the  task.  Schmidt  and  Lee  (2005)  suggest  that  the  benefits  of  distributed  over  massed 
schedules  of  practice  must  be  separated  from  the  effects  of  fatigue  present  during  training  and 
assessment  performed  at  a  later  time.  Moreover,  they  note  that  much  more  is  known  about 
continuous  as  opposed  to  discrete  tasks.  In  fact,  they  indicate  that  there  may  be  benefits  to 
massed  practice  schedules  over  distributed  schedules  for  discrete  tasks. 

Training  is  also  an  important  aspect  of  skill  acquisition  and  concerns  “how”  skills  are 
acquired.  Training  can  be  accomplished  by  practicing  the  skills  on  the  whole  task  or  part  of  the 
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task.  Often  training  on  the  whole  task  is  not  possible.  Thus,  part-task  training  incorporates  a 
subset  of  the  elements  necessary  to  acquire  the  skills.  Part-task  training  is  usually  accomplished 
by  either  segmenting  the  task  (i.e.,  decomposing  each  phase  of  the  task  into  subcomponents  to  be 
practiced),  or  fractionalizing  (i.e.,  dividing  the  task  into  two  or  more  components  that  are 
performed  together;  Wickens  &  Hollands,  2000).  Part-task  training  has  been  found  to  be 
beneficial  for  serial  tasks,  but  less  so  for  continuous  tasks  especially  where  there  are  strong 
interdependencies  among  the  parts  (Schmidt  &  Lee,  2005). 

Another  type  of  practice  that  is  crucial  for  successfully  acquiring  psychomotor  skills  is 
guided  training.  Wickens  and  Hollands  (2000)  noted  that  under  a  guided  training  paradigm,  an 
individual’s  performance  is  monitored  so  that  progress  adheres  to  the  task  requirements.  Errors 
in  learning  are  corrected  to  ensure  that  they  are  not  repeated.  Guided  training  usually  involves 
providing  feedback  to  an  individual  about  their  training  and  practice  performance  in  the  form  of 
knowledge  of  results.  Providing  knowledge  of  results  can  help  individuals  maintain  motivation 
and  enhance  performance  by  continually  reducing  the  error  between  current  levels  of 
performance  and  the  criterion  (Holding,  1987).  Wickens  and  Hollands  (2000)  suggested  that 
training  performance  is  likely  to  benefit  most  when  knowledge  of  results  is  provided 
immediately  after  a  performance  or  session. 

If  the  goal  is  efficient  psychomotor  skill  acquisition  and  retention,  then  an  understanding 
of  the  principles  of  overlearning,  schedules,  and  training  is  necessary.  That  is,  tasks  that 
reinforce  psychomotor  skills  should  be  overleamed  through  distributed  practice  sessions  so  that 
the  individual  does  not  become  fatigued  or  unmotivated.  Further,  overlearning  will  enhance  the 
probability  of  skill  retention  when  performed  at  a  later  time.  Also,  part-task  training  should  be 
conducted  by  decomposing  or  dividing  the  tasks  so  that  each  subcomponent  reinforces  the  skill 
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to  be  acquired.  Finally,  using  guided  training  by  providing  knowledge  of  results  will  help  to 
enhance  training  performance  and  initial  skill  performance. 

Medical  Simulators:  Part  of  the  Solution 

Simulation  technology  has  been  proposed  as  a  partial  solution  to  help  promote  the 
acquisition  and  retention  of  surgical  psychomotor  skills  (Kauffman,  2001;  Macintyre  &  Munro, 
1990).  There  are  significant  disadvantages  associated  with  clinical  surgical  training  today,  as 
noted  above.  However,  simulators  could  be  used  to  help  students  acquire  skills  in  a  manner  not 
possible  during  clinical  training.  Kauffman  (2001)  noted  that  surgical  simulators  provide 
students  more  time  to  practice  newly  acquired  skills.  Further,  students  can  be  required  to  reach  a 
specific  level  of  proficiency  before  being  able  to  practice  their  skills  on  a  patient.  Simulators 
also  allow  students  to  make  mistakes  and  learn  from  them  without  endangering  patients.  Perhaps 
the  most  important  advantage  of  medical  simulation  technology  is  that  it  can  help  change  the 
classic  teaching  model  of  “see  one,  do  one,  teach  one”,  to  a  more  structured  training  model  of, 
“see  one,  practice  many,  do  one,  teach  one”  (Kauffman,  1999).  Obviously,  a  new  model  that 
incorporates  additional  time  for  practice  is  advantageous  not  only  to  medical  students  and 
residents,  but  also  to  the  patients. 

Types  of  Medical  Simulators 

Medical  simulators  can  be  classified  into  three  categories:  mannequin,  virtual  reality 
(VR),  and  hybrid.  Mannequin-based  simulators  are  physical  representations  of  the  human  body 
and  its  anatomy.  One  example  of  a  mannequin-based  simulator  is  the  TraumaMan®  system  by 
Simulab,  Inc  (Figure  1).  The  TraumaMan®  system  is  a  realistic  anatomical  model  of  the  neck, 
chest,  and  abdomen  that  has  replaceable  tissue  and  fluid  components.  The  TraumaMan®  system 
is  used  for  Advance  Trauma  Life  Support  (ATLS®)  training  and  allows  students  to  perform 
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procedures  on  a  realistic  representation  of  human  anatomy.  The  main  advantage  of  mannequin- 
based  simulators  is  that  students  can  practice  with  a  simulator  just  as  they  would  with  a  genuine 
patient.  However,  mannequin-based  simulators  do  not  provide  objective  performance 
assessment.  Thus,  an  experienced  physician  must  assess  trainee  performance,  which  can  lead  to 
subjective  assessments. 


Figure  1.  Student  performing  a  thoracostomy  (chest  tube  insertion)  on  the  TraumaMan® 
by  Simulab,  Inc. 

Virtual  reality  simulators,  by  contrast,  are  based  on  computer  simulation,  as  opposed  to 
physical  models.  These  types  of  simulators  include  visual,  auditory,  and  haptic  displays  that 
enable  individuals  to  see,  hear,  and  feel  the  effects  of  their  actions.  Virtual  reality  simulators  also 
allow  students  to  interact  with  the  simulator  and  receive  objective  feedback  in  real  time  (Satava, 
2001).  Thus,  when  students  interact  with  the  simulator,  they  can  receive  immediate  feedback 
regarding  their  performance.  Virtual  reality  simulators  can  be  either  whole-task  or  part-task 
trainers.  An  example  of  a  whole-task  VR  simulator  is  the  Endoscopy  AccuTouch®  simulator 
from  Immersion  Medical,  Inc.  (Figure  2).  The  Endoscopy  AccuTouch®  simulator  has  a 
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colonoscopy  module  which  simulates  the  entire  procedure.  The  trainee  manipulates  a  facsimile 
of  a  colonoscope  while  viewing  images  of  the  colon  on  the  computer  monitor.  Haptic  feedback 
is  provided  throughout  the  procedure  and  gives  cues  as  to  the  difficulty  of  various  maneuvers 
(e.g.,  when  turns  and  loops  of  the  colon  must  be  navigated).  The  patient’s  voice  also  provides 
information  regarding  the  quality  of  the  student’s  performance.  Training  on  the  full  procedure 
has  advantages  because  it  allows  individuals  to  learn  the  proper  sequence  of  steps  and  transitions 
between  steps.  However,  when  first  learning  to  perform  a  complex  procedure,  it  may  be 
beneficial  to  learn  individual  parts  of  the  task  and  then  later  recombine  the  parts  in  order  to 
perform  the  full  procedure. 


Figure  2.  Virtual  reality  simulator  for  colonoscopy  from  Immersion  Medical,  Inc. 

One  VR  simulator  that  allows  the  individual  to  practice  parts  of  a  task,  rather  than  the 
whole  task,  is  the  MIST  VR  (system  produced  by  Mentice  AB,  see  Figure  3).  The  MIST  VR  is 
designed  for  laparoscopic  surgery;  however,  it  does  not  simulate  any  procedural  aspect  of 
laparoscopic  surgery.  Instead,  it  allows  the  trainee  to  practice  the  basic  skills  necessary  to 
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perform  minimally  invasive  surgery  in  a  variety  of  subtasks.  For  example,  one  MIST  VR  task 
requires  students  to  touch  simulated  spheres  with  the  tips  of  their  laparoscopic  instruments.  The 
main  advantage  of  learning  and  practicing  skills  on  a  part-task  trainer  is  that  the  individual  does 
not  become  overwhelmed  by  learning  the  full  procedure  all  at  once. 


Figure  3.  The  MIST  VR  simulator  for  basic  laparoscopic  skills  (by  Mentice  AB). 

Hybrid  simulators  are  those  that  incorporate  aspects  of  VR  computer-based  systems  into 
mannequin  simulators  (Satava,  2001).  These  simulators  have  certain  advantages  as  well.  Not 
only  are  individuals  able  to  interact  with  a  mannequin  that  represents  typical  human  anatomy, 
but  they  can  also  receive  some  objective  feedback  regarding  their  performance  because  the 
simulator  is  built  on  a  computer  platform.  For  example,  the  UltraSim®  system  is  a  hybrid 
ultrasound  training  simulator  that  contains  a  mannequin  abdomen,  hand  held  ultrasound  probe, 
and  computer  to  display  the  images  from  the  abdomen  (Figure  4).  As  noted  earlier,  one  of  the 
disadvantages  associated  with  mannequin-based  simulators  is  that  they  often  lack  patient  or  case 
variety.  Although  this  disadvantage  is  also  true  of  the  mannequin  portion  of  hybrid  simulators, 
the  virtual  components  help  to  over  come  this  limitation. 
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Figure  4.  The  UltraSim®  hybrid  ultrasound  training  simulator  by  MedSim,  Inc.  (photo 
provided  courtesy  of  MedSim,  Inc.) 

All  of  the  simulators  described  above  allow  students  to  practice  in  a  safe  environment. 
Thus,  students  are  free  to  make  errors  and  learn  from  their  mistakes  without  putting  patients  at 
risk.  Given  the  concerns  regarding  patient  safety  described  above,  the  development  and  use  of 
medical  simulators  may  help  improve  the  quality  of  healthcare  and  reduce  the  incidence  of 
medical  errors. 

Laparoscopy  and  Simulation  Technology 

The  majority  of  medical  VR  simulators  available  today  target  laparoscopic  skill 
acquisition.  Minimally  invasive  surgery,  via  laparoscopy,  was  quickly  embraced  during  the  late 
1980s  as  a  safer  way  to  operate  on  patients.  Laparoscopic  surgery  is  performed  using 
instruments  and  a  miniature  video  camera  that  are  inserted  through  small  incisions  in  the  body. 
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The  images  from  the  camera  are  displayed  on  a  monitor  and  allow  the  surgeon  to  view  his/her 
actions  while  performing  procedures. 

Laparoscopic  procedures  are  more  amenable  to  VR  simulation  for  many  reasons.  For 
example,  during  laparoscopic  procedures,  surgeons  can  only  manipulate  patient  anatomy 
indirectly  with  the  instruments  that  pass  through  the  incisions  in  the  body.  The  indirect  access  to 
the  patient’s  anatomy  does  not  allow  the  surgeon  to  feel  the  haptic  sensations  associated  with 
direct  contact  of  the  tissues  and  organs.  In  open  surgery,  although  surgeons  still  manipulate  the 
patient’s  anatomy  with  instruments  (e.g.,  scalpel,  forceps,  etc.),  they  can  also  directly  touch  and 
feel  the  patient’s  tissues  and  organs. 

As  Biggs  and  Srinivasan  (2002)  have  noted,  the  degree  of  difficulty  involved  in  haptic 
VR  simulation  depends  upon  whether  the  object  and  user  interaction  is  direct  (i.e.,  with  the 
fingers  and  hands)  or  accomplished  with  a  tool  (i.e.,  tip  of  a  laparoscopic  instrument).  If  the 
haptic  simulation  can  be  achieved  with  a  tool,  then  the  haptic  interface  design  is  less  complex 
and  easier  for  VR  designers  to  simulate.  The  reduced  complexity  is  due  to  point-based  haptic 
interactions  that  occur  when  the  end  of  a  tool  comes  in  contact  with  a  specific  point  on  an  object. 
In  virtual  systems,  when  the  end  point  of  a  simulated  tool  contacts  a  simulated  object,  collision 
detection  and  collision  response  rules  are  used  to  engage  the  force  feedback  system  and  constrain 
the  user’s  movement  (Biggs  &  Srinivasan,  2000).  The  collision  algorithms  are  fairly  easy  to 
render  in  a  point-based  haptic  system  and  therefore  lend  themselves  to  laparoscopic  procedures 
because  much  of  the  activity  occurs  through  single  points  of  contact. 

The  visual  representation  of  the  surgeon’s  movements  and  manipulation  of  anatomy  is 
another  major  issue  concerning  laparoscopic  VR  simulators.  Designers  must  represent  3D 
anatomy  and  on  a  2D  display  monitor.  Although  this  is  a  challenge,  it  is  not  as  difficult  as 
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attempting  to  realistically  represent  the  anatomy  in  3D.  Thus,  laparoscopic  procedures  are 
probably  more  accessible  to  designers  of  VR  simulators. 

It  is  indeed  fortunate  that  there  are  fewer  technical  challenges  simulating  laparoscopic 
procedures  because  this  type  of  surgery  is  more  challenging  for  the  surgeon.  The  skills  required 
to  perform  laparoscopic  surgery  are  very  different  from  that  of  open  surgery.  Further,  because 
laparoscopic  surgery  was  quickly  adopted,  the  difficulties  associated  with  the  procedures  were 
not  fully  investigated.  Deziel  et  al.  (1993)  surveyed  U.S.  hospitals  on  their  major  operative 
complication  rates  for  laparoscopic  cholecystectomy  (i.e.,  gall  bladder  removal)  and  found  that 
nearly  60  percent  of  reported  injuries  were  to  the  patient’s  bile  duct,  the  most  common 
complication.  The  other  most  common  injuries  were  to  vascular  and  bowel  tissue.  Deziel  et  al. 
suggested  that  despite  higher  complication  rates  with  laparoscopy,  the  procedure  is  generally 
safe.  Moore  and  Bennett  (1995)  also  assessed  laparoscopic  cholecystectomy  complication  rates 
and  found  that  most  complications  result  from  relatively  inexperienced  surgeons  (i.e., 
complications  usually  occur  within  the  first  thirty  cases).  Moore  and  Bennett  performed  a 
regression  analysis  on  their  data  and  noted  that  bile  duct  injuries  could  be  expected  to  decrease 
from  1.7%  for  the  first  case  to  0.17%  after  completion  of  50  cases.  Moore  and  Bennett  also 
noted  that  the  speed  with  which  the  procedure  was  implemented  in  the  late  1980s,  along  with  the 
absence  of  research  on  training,  and  risks  and  benefits  to  the  patient,  were  likely  contributors  to 
the  higher  complication  rates. 

Although  laparoscopic  procedures  seem  to  have  higher  complication  rates  for  less 
experienced  surgeons,  it  is  still  the  preferred  surgical  method  for  many  procedures.  One  reason 
for  its  wide  acceptance  rests  with  the  benefits  for  the  patient  (Munz  et  al.,  2004).  The  most 
important  benefit  is  a  lower  risk  of  complications  following  surgery  in  patients  who  undergo 
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laparoscopic  procedures.  Another  advantage  for  patients  is  the  cosmetic  appeal  of  laparoscopy. 
Patients  typically  have  less  visible  scarring.  Patients  also  have  a  quicker  recovery  time  after 
laparoscopic  procedures.  Although  laparoscopic  procedures  and  techniques  have  resulted  in 
important  benefits  for  the  patients,  surgeons  face  more  challenges  when  performing  laparoscopic 
surgery. 

Gallagher  and  Smith  (2003)  have  described  some  of  the  difficulties  surgeons  face  with 
the  minimally  invasive  technique.  In  particular,  laparoscopy  creates  new  perceptual,  spatial,  and 
psychomotor  demands  for  the  surgeon.  Gallagher  and  Smith  (2003)  and  Gallagher  et  al.  (2003) 
point  out  that  perceptually,  the  surgeon  must  translate  information  that  is  presented  on  a  2D 
monitor  into  a  3D  mental  model  of  the  human  anatomy.  Others  have  also  noted  that  the  surgeon 
performing  laparoscopy  must  compensate  for  navigating  in  a  3D  environment  with  a  2D  monitor 
by  using  depth  cues  other  than  stereopsis  (Ahlberg  et  al.,  2002). 

The  fulcrum  effect  is  another  perceptual  problem  that  surgeons  encounter  during 
laparoscopic  surgery.  Because  laparoscopic  instruments  pass  through  into  small  incisions  in  the 
body,  the  amount  of  space  available  to  move  the  instruments  is  limited.  Further,  the  instruments 
must  be  pivoted  around  the  insertion  point  in  the  patient’s  body,  thereby  creating  a  fulcrum. 
When  the  instrument  is  moved  in  one  direction,  it  appears  as  if  it  is  moving  in  the  opposite 
direction  on  the  monitor.  Thus,  the  fulcrum  effect  refers  to  the  perceptual  incongruency  between 
the  perceived  physical  movement  and  the  representation  of  that  movement  on  the  display 
monitor.  Gallagher  et  al.  (2003)  noted  that  the  fulcrum  effect  creates  psychomotor  difficulties 
for  laparoscopic  surgeons.  Further,  the  incongruence  between  the  image  seen  on  the  monitor  and 
the  surgeon’s  mental  model  of  what  the  anatomy  should  look  like  can  also  be  a  problem.  For 
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example,  the  patient’s  anatomy  can  look  distorted  depending  upon  where  the  laparoscope  is 
located  (i.e.,  if  the  laparoscope  is  too  close  or  too  far  away  from  the  anatomy). 

Ergonomics  is  also  an  issue  in  laparoscopic  surgery.  Hanna  et  al.  (1998)  noted  that  there 
can  be  problems  with  the  location  of  the  display  monitor.  Often  the  monitor  is  placed  to  the  side 
of  the  surgeon’s  field  of  view  because  laparoscopic  surgery  is  performed  in  operating  rooms  that 
were  traditionally  built  to  accommodate  open  surgery  and  thus  lack  the  space  required  for  the 
laparoscopic  equipment.  Hanna  et  al.  conducted  a  study  in  which  endoscopic  knot  tying  and 
overall  knot  quality  in  a  box-trainer  (see  below)  were  assessed  with  different  monitor  positions. 
They  found  that  when  the  monitor  was  placed  slightly  below  the  head  and  close  to  the  surgeon’s 
hands,  knot-tying  time  decreased  and  overall  quality  of  the  knot  increased.  Hanna  et  al. 
concluded  that  this  monitor  position  was  associated  with  better  surgical  performance.  However, 
they  did  not  report  the  amount  of  prior  endoscopic  knot-tying  experience  their  participants  had. 

It  is  possible  that  their  participants  may  have  had  extensive  practice  on  a  box  trainer,  which 
traditionally  places  the  monitor  close  to  the  hands  and  directly  in  front  of  the  head.  In  an  actual 
operating  room,  positioning  the  monitor  directly  in  front  of  the  surgeon  and  closer  to  his/her 
hands  may  not  be  possible  due  to  the  need  to  access  the  patient  in  the  case  of  an  emergency. 

To  overcome  some  of  the  difficulties  associated  with  laparoscopic  procedures,  surgeons 
have  turned  to  a  variety  of  training  methods  to  facilitate  skill  acquisition  outside  the  OR.  The 
training  methods  are  tied  to  two  different  classes  of  simulators:  box  trainers,  which  are  similar  to 
mannequin  simulators,  and  VR-based  systems. 

Laparoscopic  Box  Trainers 

Box  trainers  have  been  used  to  help  surgeons  acquire  the  skills  necessary  to  perform 
laparoscopic  surgery.  They  usually  consist  of  a  box-like  structure  with  a  camera  and  light  source 
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that  displays  the  images  inside  the  box  trainer  on  a  monitor  (Figure  5).  An  opaque  covering 
prohibits  direct  viewing  inside  the  box,  thus,  allowing  only  the  images  on  the  monitor  to  be  used 
as  cues.  Standard  laparoscopic  instruments  are  then  used  to  manipulate  various  objects  within 
the  box  trainer.  The  tasks  typically  associated  with  a  box  trainer  include  placing  objects  on  a 
pegboard,  cutting  fabric,  and  applying  clips.  There  is  an  assumption  that  the  skills  learned  using 
the  box  trainer  will  transfer  to  an  actual  laparoscopic  procedure.  However,  this  assumption  has 
not  been  extensively  or  objectively  confirmed  in  the  literature. 


Figure  5.  The  laparoscopic  box  training  system  by  3-D  Technical  Services,  Inc. 

Two  examples  of  research  with  box  trainers  suggest  that  evidence  for  the  transfer  of 
laparoscopic  skills  is  lacking.  Derossis  et  al.  (1998)  used  a  box  trainer  to  develop  objective 
measures  of  laparoscopic  skills.  They  asked  surgical  residents  to  perform  a  series  of 
laparoscopic  box  trainer  tasks  (e.g.,  peg  board  transfers,  cutting,  and  suturing)  and  then  assessed 
their  performance  speed  and  precision.  Derossis  et  al.  found  that  those  who  were  further  along 
in  their  residency,  performed  significantly  better  than  newer  residents.  They  suggested  that  box 
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trainers  can  provide  objective  measures  of  laparoscopic  skills  because  they  can  differentiate 
performance  among  years  in  residency  and  amount  of  practice.  Another  interesting  finding  from 
this  study  concerns  performance  on  the  individual  tasks.  When  performance  speed  and  precision 
of  laparoscopic  surgeons  was  compared  to  nonlaparoscopic  surgeons,  laparoscopic  surgeons 
significantly  outperformed  nonlaparoscopic  surgeons  on  only  three  of  the  seven  box  trainer 
tasks:  pegboard  patterns,  pattern  cutting,  and  intracorporeal  knot  tying.  The  tasks  which  did  not 
produce  a  significant  difference  between  the  groups  were  clip  application,  placement  of  ligating 
tube,  mesh  placement  over  a  defect,  and  extracorporeal  knot  tying.  This  finding  suggests  that  the 
latter  set  of  box  trainer  tasks  may  not  have  been  difficult  or  unique  enough  to  laparoscopy  to 
distinguish  between  the  groups.  Also,  because  all  participants  were  only  assessed  on  a  box 
trainer,  it  is  not  known  whether  these  results  would  be  replicated  in  real  laparoscopic  surgery. 

One  of  the  more  challenging  laparoscopic  tasks  is  intracorporeal  suturing,  or  the  ability  to 
tie  knots  within  the  patient’s  body.  Komdorffer  and  his  colleagues  (in  press)  used  a  box  training 
system  to  train  residents  on  this  skill.  Importantly,  the  residents  were  required  to  perform  the 
knot-tying  task  in  a  porcine  laboratory  (i.e.,  on  a  live  pig)  before  and  after  training.  The  residents 
trained  on  the  Storz  Endoscopy  videotraining  system  that  included  foam  pads  for  suturing. 
Komdorffer  et  al.  observed  that  the  residents  needed  1 5 1  minutes  of  practice  on  the  box  trainer, 
on  average,  to  reach  performance  levels  comparable  to  those  of  an  expert  group  on  that  training 
system.  Results  from  the  porcine  tests  showed  that  the  trainees  achieved  significant  gains  in 
performance  from  the  pretest  to  posttest  on  several  measures,  but  they  did  not  achieve  the  same 
levels  exhibited  by  the  experts  in  the  porcine  lab.  The  authors  concluded  that  the  box  training 
system  does  produce  positive  transfer  to  genuine  operating  procedures,  but  at  a  lower  level  than 
what  would  be  expected  from  expert  surgeons. 
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Macmillan  and  Cuschieri  (1999)  used  a  different  box  trainer  known  as  the  Advanced 
Dundee  Endoscopic  Psychomotor  Tester  (ADEPT)  in  an  attempt  to  predict  how  well  innate 
surgical  abilities  correlate  with  surgical  performance.  ADEPT  is  an  advanced  box  trainer 
because  it  is  computer  based,  thus  allowing  the  software  to  dictate  the  training  regimen  and 
record  performance  indices.  However,  the  tasks  to  be  performed  are  similar  to  traditional  box 
trainer  tasks  and  primarily  require  object  manipulation.  Ten  surgical  residents  were  given 
approximately  one  hour  of  training  on  the  ADEPT  simulator.  Expert  surgeons  then  rated  the 
clinical  skills  of  the  trainee  surgeons.  Macmillan  and  Cuschieri  (1999)  found  a  significant 
correlation  ( r  =  .79)  between  participant  success  rate  on  ADEPT  and  clinical  skills  assessed  on 
actual  laparoscopic  surgery. 

Box  trainers,  though  still  widely  used,  are  now  giving  way  to  VR-based  simulators.  Box 
trainers  are  relatively  static,  in  that  only  a  small  number  of  elements  can  be  changed  for  further 
practice.  Virtual  reality  simulators,  however,  allow  for  a  more  dynamic  interaction  (e.g., 
increased  case  variety,  real-time  interaction,  etc.). 

Virtual  Reality  Laparoscopic  Trainers 

Virtual  reality  trainers  have  an  important  advantage  over  box  trainers  because  they  allow 
a  wider  range  of  performance  indices  to  be  quantified  and  recorded.  The  Minimally  Invasive 
Surgical  Trainer  (MIST  VR)  is  the  most  widely  used  and  researched  VR  trainer.  It  consists  of  a 
PC  with  a  laparoscopic  interface  that  can  track  and  translate  instrument  movements  into  3D 
graphics  displayed  on  the  monitor  (Sutton  et  al.,  1997).  The  MIST  VR  has  five  main 
components:  tutorial,  training,  examination,  analysis,  and  configuration.  The  training  component 
consists  of  six  different  tasks  including  target  placement,  target  diathermy,  tool  withdrawal  and 


insertion. 
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MIST  VR  Learning  Curves  and  Sensitivity 

Because  the  MIST  VR  has  been  studied  extensively,  a  number  of  performance  criteria  are 
available  for  review.  Chaudhry  et  al.  (1999)  assessed  the  learning  curves  on  six  MIST  VR  tasks 
over  10  trials  for  1 1  surgeons  and  7  nonsurgeons  (hospital  staff).  Their  results  showed  that 
although  the  overall  scores  of  experienced  surgeons  were  significantly  better  on  all  tasks 
(demonstrating  that  the  MIST  VR  has  sensitivity),  there  were  no  significant  differences  in  the 
learning  curves  or  familiarization  rates  between  the  two  groups.  All  participants  took  up  to  three 
sessions  to  familiarize  themselves  with  the  tasks  and  learn  the  processes,  including  how  to 
correctly  manipulate  the  instruments.  However,  another  study  of  learning  curves  associated  with 
MIST  VR  revealed  significant  differences  in  the  familiarization  curves  for  surgeons  with 
different  levels  of  laparoscopic  experience  (master,  intermediate,  and  beginner).  Grantcharov  et 
al.  (2003)  assessed  the  surgeons  on  all  six  MIST  VR  tasks  10  separate  times  within  a  one-month 
period.  They  found  that  master  surgeons  had  a  steeper  learning  curve  for  the  tasks  than  surgeons 
with  less  laparoscopic  experience.  The  familiarization  curves  for  master  surgeons  differed 
significantly  from  those  of  intermediate  and  beginner  surgeons  on  several  measures  including 
time,  errors,  and  unnecessary  instrument  movements.  Intermediate  surgeons  were  also  found  to 
have  more  rapid  familiarization  curves  for  the  tasks  than  beginner  surgeons.  Brunner  et  al. 

(2004)  studied  skill  acquisition  with  MIST  VR  over  an  extended  period  of  time  (i.e., 
approximately  7  hours).  An  analysis  of  the  learning  curves  indicated  that  skills  were  acquired  in 
successive  stages  and  that  even  after  7  hours  and  30  task  repetitions,  improvements  in 
performance  had  still  not  reached  their  final  plateau.  Collectively,  these  studies  with  MIST  VR 
demonstrate  that  the  simulator  is  sensitive  and  can  differentiate  between  individuals  with 
different  levels  skill  and  surgical  experience. 
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Overcoming  the  Fulcrum  Effect 

As  noted  above,  the  fulcrum  effect  refers  to  the  incongruency  between  real  and  displayed 
movements  that  occur  during  laparoscopic  surgery.  Gallagher  et  al.  (1999)  assessed  the  degree 
to  which  MIST  VR  could  help  individuals  overcome  the  fulcrum  effect  with  participants  who 
had  no  prior  experience  with  endoscopy  procedures.  Half  were  trained  on  the  MSIT  VR  and  the 
other  half  served  as  the  control  group.  After  completing  MSIT  VR  training,  all  participants  were 
asked  to  perform  a  cutting  task  with  a  box  trainer.  Their  results  demonstrated  that  the 
individuals  trained  on  MIST  VR  made  significantly  more  correct  incisions  in  a  standard  box 
training  task  when  compared  to  the  control  group.  Gallagher  et  al.  (1999)  suggested  that  the 
superior  performance  of  the  MIST  VR  trained  group  could  be  attributed  to  their  being  able  to 
acclimate  to  the  fulcrum  effect  more  quickly.  Jordan  et  al.  (2000)  also  studied  the  fulcrum  effect 
with  the  MIST  VR  and  compared  it  to  standard  laparoscopic  training  conditions  (i.e.,  with  a  box 
trainer).  They  assigned  participants  to  one  of  three  training  conditions:  1)  MIST  VR,  2)  a  box 
trainer  with  randomly  alternating  normal  and  y-axis  inverted  views,  and  3)  a  box  trainer  with 
normal  laparoscopic  views.  Participants  were  given  a  pretest  on  a  box  trainer  cutting  task,  six 
training  tasks  during  one  of  the  three  conditions,  and  then  a  posttest  on  the  same  box  trainer  task. 
They  found  that  participants  trained  on  MIST  VR  made  significantly  more  correct  incisions  on 
the  posttest,  followed  by  those  in  the  randomly  alternating  condition,  and  then  those  in  the 
normal  laparoscopic  condition.  Jordan  et  al.  (2000)  suggested  that  training  on  MIST  VR  helped 
individuals  acclimate  to  the  fulcrum  effect  more  quickly  than  training  on  a  standard  box  trainer. 
However,  the  superior  performance  of  those  trained  on  MIST  VR  described  in  the  studies  above 
has  only  been  demonstrated  on  box  trainer  tasks.  Obviously,  the  critical  test  for  MIST  VR 
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training  requires  that  the  skills  learned  in  the  simulator  transfer  to  the  operating  room  with  real 
laparoscopic  procedures  performed  on  genuine  patients. 

Transfer  of  MIST  VR  Acquired  Skill 

To  date,  there  have  been  only  two  attempts  to  validate  transfer  of  MIST  VR  acquired 
skills  to  real  laparoscopic  procedures.  In  the  first  study,  the  goal  was  to  determine  whether  skills 
acquired  on  MIST  VR  would  transfer  to  a  real  laparoscopic  procedure  performed  in  a  porcine 
laboratory.  Alhberg  et  al.  (2002)  used  fourth-year  medical  students  who  had  no  prior  endoscopic 
experience.  Half  of  their  participants  received  two  training  sessions  on  MIST  VR  for  a  total  of 
three  hours,  and  the  other  half  received  no  training.  All  participants  then  performed  an 
appendectomy  on  a  pig.  The  surgical  sessions  were  recorded  and  scored  by  three  observers  for 
five  relevant  performance  metrics  (e.g.,  grasping  bowel,  cutting  ligature,  loop  ligation,  etc.). 
Alhberg  et  al.  (2002)  did  not  find  any  significant  performance  differences  between  the  MIST  VR 
and  untrained  groups.  The  authors  suggested  that  absence  of  any  performance  benefit  for  MIST 
VR  may  have  been  due  to  the  lack  of  haptic  feedback.  Thus,  MIST  VR  may  facilitate  hand-eye 
coordination,  but  fails  to  provide  the  haptic  sensations  important  for  real  laparoscopic 
procedures.  Therefore,  when  MIST  VR  trained  students  had  to  perform  the  pig  appendectomy, 
they  may  have  been  familiar  with  the  visual  issues  associated  with  laparoscopic  surgery,  but  had 
to  acclimate  to  the  haptic  sensations  associated  with  manipulating  tissue. 

Unlike  Alhberg  et  al.  (2002),  Seymour  and  his  colleagues  (2002)  attempted  to  determine 
whether  MIST  VR  acquired  skills  would  transfer  to  genuine  laparoscopic  procedures  performed 
on  human  patients.  Seymour  et  al.  (2002)  used  surgical  residents  who  were  in  their  first  to  fourth 
year  of  training  and  assessed  their  fundamental  abilities  (i.e.,  visuospatial,  perceptual,  and 
psychomotor  abilities).  Half  were  assigned  to  MIST  VR  plus  standard  training  and  the  other  half 
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to  only  standard  training.  The  authors  noted  that  the  standard  training  was  appropriate  for  their 
level  of  residency,  but  did  not  provide  exact  details  regarding  the  nature  of  that  training.  The 
group  that  received  training  on  MIST  VR  completed  a  single  training  session  lasting  one  hour. 
Following  training,  all  residents  watched  a  video  about  laparoscopic  cholecystectomy  and  then 
performed  the  procedure  under  the  supervision  of  a  surgeon.  All  of  the  procedures  were 
recorded  and  assessed  by  surgeons  who  were  blind  to  each  participant’s  condition.  Seymour  et 
al.  (2002)  found  that  residents  who  trained  on  MIST  VR  and  received  standard  laparoscopic 
training,  completed  the  laparoscopic  cholecystectomy  in  29%  less  time  than  their  counterparts  in 
the  standard  only  condition.  Those  trained  on  MIST  VR  also  committed  fewer  errors  during 
actual  gall  bladder  removal  than  those  in  the  standard  only  group.  Seymour  et  al.  argued  that 
training  on  a  VR  simulator  can  produce  transfer  of  laparoscopic  skills  to  the  operating  room. 
However,  it  is  important  to  remember  that  their  VR  group  received  standard  training  in  addition 
to  MIST  VR  training.  Thus,  one  cannot  determine  the  degree  of  improvement  associated  with 
MIST  VR  training  relative  to  another  group  receiving  another  form  of  training  for  comparable 
duration. 

Conclusion 

The  research  regarding  training  on  MIST  VR  demonstrates  some  benefits  for 
laparoscopic  skill  acquisition.  However,  the  extent  of  those  benefits  is  still  not  known.  To  date, 
there  have  been  no  studies  that  suggest  training  solely  on  MIST  VR  will  enable  individuals  to 
acquire  necessary  laparoscopic  skills  compared  to  other  training  devices  (e.g.,  using  a  box 
trainer).  The  degree  of  skill  transferability  acquired  with  surgical  simulators  is  just  one  problem 


medical  educators  must  consider. 


Medical  Simulators  25 


Another  problem  with  incorporating  simulators  into  medical  school  training  or  residency 
curricula  is  that  many  of  the  simulators  have  not  achieved  a  sufficiently  high  degree  of  realism. 
Satava  (1993)  suggested  that  medical  simulators  must  have  adequate  fidelity,  objective  images 
that  behave  according  to  their  anatomical  properties  (e.g.,  tissue  that  is  displaced  properly  when 
touched  or  grasped),  an  element  of  interactivity,  and  haptic  qualities.  The  MIST  VR,  for 
example,  is  a  part-task  trainer  using  only  abstract  tasks.  The  MIST  VR  also  lacks  haptic  force 
feedback,  yet  it  is  still  the  most  widely  researched  surgical  simulator  on  the  market.  Satava 
(2001)  noted  that  designers  of  medical  simulators  must  still  address  the  tradeoff  between  visual 
fidelity  and  interactivity.  That  is,  efforts  to  maximize  visual  fidelity  are  often  achieved  at  the 
expense  of  interactivity  and  vice  versa.  However,  Satava  (2001)  noted  that  the  biggest  challenge 
associated  with  training  individuals  on  medical  simulators  concerns  the  optimal  ways  of 
incorporating  them  into  a  curriculum  where  they  can  be  used  for  skills  training  and  evaluation. 

Another  criticism  of  medical  simulators  is  that  they  may  dehumanize  the  healthcare 
profession,  which  seeks  to  care  for  individuals  (Issenberg  et  al.,  1999).  Some  critics  fear  that  by 
focusing  on  computerized  machines  (i.e.,  simulators)  for  a  portion  of  medical  training,  students 
may  be  missing  the  importance  of  personalized  patient  care.  However,  as  Issenberg  et  al.  (1999) 
suggest,  patient  care  can  still  be  learned  at  a  later  time,  after  students  have  been  able  to 
familiarize  themselves  with  medical  procedures  and  pathology.  Thus,  simulators  in  medical 
education  must  achieve  a  degree  of  realism  that  is  currently  lacking  and  their  use  needs  to  be 
balanced  with  real  patient  interactions. 

Although  many  medical  simulators  are  now  being  used  in  education  and  training,  their 
efficacy  must  still  be  considered  (Satava,  2001).  For  a  simulator  to  be  truly  effective,  it  must 
exceed  the  benefits  currently  provided  by  standard  methods  of  training.  Therefore,  educators 
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must  be  aware  of  the  limitations  associated  with  using  simulators  to  train  students.  Often  using 
simulation  as  a  training  method  enables  individuals  to  acquire  only  some  of  the  skills  associated 
with  a  procedure  or  task.  For  example,  when  students  train  on  MIST  VR  they  are  able  to  acquire 
the  fundamental  psychomotor  skills  associated  with  laparoscopic  surgery.  However,  because 
MIST  VR  is  only  a  part-task  trainer,  students  must  also  familiarize  themselves  with  relevant 
anatomy  and  the  details  of  specific  laparoscopic  procedures.  Educators  must  assess  the  benefits 
and  limitations  of  simulators  and  make  decisions  regarding  their  appropriateness  and  place  in  the 
curriculum. 

The  decision  to  incorporate  simulation  training  into  a  curriculum  is  not  only  dependent 
upon  the  potential  training  benefits,  but  also  the  financial  costs  involved.  Satava  (2001)  noted 
the  financial  challenges  associated  with  using  simulation  for  training.  For  example,  because  VR 
simulators  are  built  on  a  computer  platform,  the  hardware  and  software  required  can  be  very 
expensive.  Thus,  the  types  of  medical  simulators  that  are  currently  available  are  somewhat 
simplistic  and  address  procedures  that  are  relatively  easy  to  simulate.  Satava  (2001)  suggests 
that  three  areas  must  be  addressed  for  simulation  to  be  a  cost  effective  training  method.  First, 
simulators  should  be  designed  for  many  different  types  of  surgical  specialties.  Second,  they 
should  accommodate  individuals  with  different  skill  levels  (i.e.,  students,  residents,  attendings, 
etc.).  Third,  simulators  should  not  only  be  used  for  education  and  training,  but  also  for  planning 
complex  surgical  procedures.  As  Satava  noted,  the  cost  effectiveness  of  simulators  in  medical 
until  costs  decrease,  simulation  will  likely  be  available  for  training  on  less  complex  procedures. 

Further  research  is  also  needed  regarding  the  ideal  way  to  use  simulators  within  the 
curriculum.  Most  of  the  research  regarding  simulation  training  is  not  theoretically  driven. 

Theory  of  learning,  training,  and  skill  development  are  necessary  to  determine  the  effectiveness 
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of  simulation-based  training.  Many  principles  of  skill  acquisition  have  yet  to  be  addressed  in  the 
research  and  development  of  simulators  for  medical  training  and  education.  Schedules  of 
practice,  guided  training,  overlearning,  and  types  of  trainers  all  need  to  be  considered  in  future 
research  to  maximize  the  benefits  of  simulation-based  training. 

Learning  and  skill  acquisition  are  issues  that  must  be  continually  evaluated  in  simulation- 
based  medical  training.  However,  simulation  holds  a  lot  of  promise  for  training  future  health 
care  professionals.  By  conducting  theory-driven  research  and  following  the  principles  of 
learning  and  skill  acquisition,  individuals  in  medicine  may  one  day  be  able  to  acquire  and 
practice  the  skills  they  need  before  ever  touching  a  real  patient. 
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Within  the  last  five  years,  several  events  have  taken  place  that  will  have  a  profound  effect  on  the 
practice  of  medicine  and  patient  safety.  First,  medical  virtual  reality  simulators  have  been 
developed  for  training  minimally  invasive  and  other  forms  of  surgical  procedures.  Second, 
evidence  is  beginning  to  show  that  surgeons  who  train  with  this  technology  are  more  skillful 
when  they  operate  on  genuine  patients.  Third,  several  regulatory  agencies  will  soon  begin  to 
require  this  technology  for  training  and  certification.  At  present,  most  medical  virtual  reality 
simulators  have  been  developed  with  little  or  no  human  factors  involvement,  but  the  need  for 
human  factors  knowledge  and  expertise  is  paramount.  In  this  paper,  I  describe  what  I  perceive  to 
be  the  top  10  areas  where  human  factors  professionals  can  contribute  to  the  evolution  and 
adoption  of  this  new  technology. 

INTRODUCTION 


The  use  of  computer-based  simulators  for  training 
has  a  long  history  within  the  human  factors  profession. 
Research  addressing  flight,  ATC,  driving,  power  plant, 
and  command  and  control  simulators  is  so  common 
within  the  pages  of  our  literature  that  we  almost  take  this 
technology  for  granted.  However,  there  is  still  one  high- 
risk  profession  in  which  simulation-based  technology 
has  been  conspicuously  absent  -  medicine. 

Although  medical  simulation  devices  have  been 
around  since  the  1940s,  most  of  them  have  been  little 
more  than  physical  models  with  very  limited 
functionality.  That  began  to  change,  however,  in  the 
early  1990s  as  a  result  of  three  critical  developments 
(Scerbo,  2005).  First,  the  Visible  Human  project 
sponsored  by  the  National  Library  of  Medicine  produced 
virtual  representations  of  a  patient’s  anatomy  and 
systems  (Satava,  2001).  Second,  a  new  procedure  was 
introduced  for  performing  cholecystectomy  (i.e.,  gall 
bladder  removal)  by  which  the  surgeon  makes  small 
incisions  in  the  abdomen,  inserts  a  small  video  camera 
and  two  surgical  instruments,  and  performs  the  operation 
from  outside  the  body.  By  the  end  of  the  decade,  this 
minimally  invasive  (or  laparoscopic)  process  became  the 
standard  for  a  growing  number  of  surgical  procedures. 
Third,  haptic  force  feedback  rendering  systems  that 
enable  users  to  touch  and  manipulate  objects  in  virtual 
space  were  developed  and  refined.  Collectively,  these 
events  made  it  feasible  to  develop  a  virtual  reality  (VR) 
surgical  simulator  by  using  facsimiles  of  laparoscopic 
instruments  interfaced  with  force  feedback  systems  and 
visual  graphics  to  permit  interaction  with  virtual  organs. 

In  2002,  Seymour  at  al.  published  the  results  of  the 
first  double-blind  experiment  comparing  training  via  the 
traditional  apprenticeship  approach  to  surgery  and 
training  with  a  VR  simulator.  The  simulator  they  studied 
was  the  MIST®  VR  system  which  uses  abstract 


representations  of  laparoscopic  tasks  to  train  the 
fundamental  psychomotor  skills  needed  to  perform  the 
procedures.  Their  results  showed  that  residents  who 
trained  on  the  simulator  needed  30%  less  time  to 
perform  a  genuine  procedure  than  those  trained 
according  to  the  traditional  method.  Moreover,  those 
who  trained  on  the  VR  system  were  less  likely  to  injure 
the  gall  bladder,  bum  nontarget  tissue,  or  have  the 
attending  surgeon  step  in  and  take  over. 

More  recently,  Gallagher  and  Cates  (2004a) 
reported  on  their  experience  with  a  different  surgical 
procedure  for  patients  who  have  plaque  build-up  in  the 
carotid  artery.  Because  the  risk  of  stroke  is  extremely 
high  for  patients  with  this  condition,  a  new  procedure 
was  developed  in  which  a  wire  stent  is  placed  in  the 
carotid  artery  to  reduce  the  possibility  of  an  embolism 
and  improve  blood  flow.  The  procedure  is  both 
complicated  and  risky.  In  essence,  it  requires  the 
physician  to  insert  various  catheters  into  the  body  and 
manipulate  them  to  reach  the  carotid  artery,  deploy  a 
catch  basket,  break  up  the  plaque,  place  the  stent,  and 
then  remove  the  basket  along  with  the  loosened  plaque. 
Until  recently,  there  were  very  few  physicians  who  could 
perform  the  procedure  and  further,  it  was  not 
reimbursable  through  insurance  companies. 

In  2000,  researchers  at  the  Center  for  Integration  of 
Medicine  and  Innovative  Technology  began  work  on 
what  would  become  the  VIST®  Vascular  Interventional 
System  for  Training.  This  VR  simulator  uses  physical 
models  of  the  catheters  and  the  vascular  system  to 
generate  graphical  images  that  can  reproduce  the 
appearance  and  physiological  dynamics  depicted  on  the 
cardiologist’s  X-Ray  display.  The  simulator  also  records 
a  variety  of  performance  metrics  (e.g.,  completion  time, 
accuracy,  use  of  contrasting  agent,  etc.)  that  can  be  used 
to  assess  skill  acquisition. 

In  March  2004,  Gallagher  and  Cates  (2004b)  met 
with  the  US  Food  and  Drug  Administration  to  present 
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evidence  for  the  efficacy  of  VR-based  simulation  for 
skills  training  and  assessment.  The  following  month,  the 
FDA  voted  to  approve  the  carotid  stent  procedure 
accompanied  by  simulation-based  training.  As  Scerbo 
(2005)  noted,  physicians  wishing  to  perform  the 
procedure  would  have  to  meet  a  set  of  training  criteria 
on  the  simulator  established  by  a  normative  group  of 
physicians  who  have  extensive  experience  with  the 
procedure.  After  they  demonstrated  competency  on  the 
simulator,  they  would  be  allowed  to  perform  the 
procedure  on  patients.  A  formal  experimental  trial 
comparing  VR-trained  and  non-VR-trained  physicians 
on  the  procedure  is  currently  underway. 

This  action  taken  by  the  FDA  is  an  historical  event 
of  unimaginable  proportions.  It  is  the  first  time  in  the 
history  of  medicine  that  performance-based  competency 
measures  will  determine  who  can  and  cannot  perform  a 
medical  procedure.  It  is  also  the  first  time  the  FDA  has 
approved  a  device  along  with  a  requirement  that  dictates 
how  physicians  must  treat  their  patients.  Consequently, 
the  governing  boards  of  several  medical  specialties  (e.g., 
American  College  of  Surgeons)  are  now  giving  serious 
consideration  to  simulation-based  training  and 
competency  assessment  in  their  curricula  (Healy,  2002). 

Gallagher  and  his  colleagues  (2005)  argue  that 
medical  VR  simulators  are  creating  a  paradigmatic  shift 
in  how  medicine  will  be  taught  and  practiced  in  the 
future.  Although  it  is  exciting  to  see  these  changes 
taking  place  in  medicine,  it  is  also  sobering  to  realize 
that  all  of  this  has  been  accomplished  with  little  or  no 
human  factors  involvement.  There  were  no  human 
factors  designers  on  the  development  teams  for  either  of 
the  two  simulator  systems  responsible  for  these  events, 
(i.e.,  MIST®  VR  and  VIST®  VR). 

Clearly,  the  knowledge  and  experience  human 
factors  professionals  have  acquired  with  simulation- 
based  technology  across  numerous  domains  suggest  that 
our  profession  has  much  to  offer  for  those  working  in  the 
medical  profession.  In  the  remainder  of  the  paper,  I 
describe  what  I  consider  to  be  the  10  most  important 
areas  where  human  factors  can  and  should  contribute  to 
the  evolution  of  medical  and  surgical  simulation 
technology. 

TOP  10  AREAS  FOR 
HUMAN  FACTORS  CONTRIBUTIONS 

1)  Training  paradigms 

Recently,  the  American  Medical  Association’s 
Accreditation  Council  for  Graduate  Medical  Education, 
the  organization  that  accredits  teaching  hospitals  in  the 
U.S.,  established  an  80-hour/week  limit  on  the  number 
of  hours  medical  residents  can  work  (AMA,  2003). 

These  new  restrictions  represent  about  a  10%  -  30% 


decrease  in  the  typical  working  hours  of  residents  and 
medical  school  faculty  members  are  greatly  concerned 
about  their  ability  to  provide  adequate  training  within 
this  restricted  time  frame. 

VR  simulators  may  provide  a  partial  solution  to  this 
problem  because  they  allow  trainees  to  learn  procedures 
largely  on  their  own.  However,  designers  of  medical 
simulators  and  curricula  alike  are  now  investigating 
methods  to  optimize  training.  Human  factors  researchers 
can  offer  guidance  on  principles  of  knowledge  and  skill 
acquisition  that  address  task  sequencing,  part  and  whole 
task  training,  and  hierarchical  and  integrative  approaches 
(Proctor  &  Dutta,  1995).  Also,  we  understand  how  to 
assess  transfer  of  training  and  conduct  task  analyses  to 
identify  procedural  components  that  promote  the 
development  of  automatic  skills  and  more  efficient 
performance  (Schneider,  1985). 

2)  Simulator  fidelity 

A  major  issue  facing  developers  of  medical  VR 
systems  concerns  simulator  fidelity.  Designers  continue 
to  debate  over  how  much  fidelity  is  necessary.  Human 
factors  researchers  have  a  history  of  studying  simulator 
fidelity  issues  (Hays  &  Singer,  1989).  We  understand 
that  simulator  fidelity  differs  along  physical  and 
functional  dimensions.  Further,  the  degree  of  fidelity 
should  follow  from  an  analysis  of  training  requirements. 
In  addition,  the  costs  associated  with  building  and 
training  in  a  high  fidelity  simulator  must  be  weighed 
against  the  degree  of  transfer  from  the  simulated  to 
operational  environment.  Further,  many  high  fidelity 
simulators  may  have  negligible  performance  benefits 
over  more  modest  systems  (Lintem,  1991).  Clearly,  we 
can  offer  much  guidance  on  nature  of  simulator  fidelity 
as  it  relates  to  medical  procedures. 

3)  Virtual  environments  (VEs) 

Human  factors  researchers  have  extensive 
experience  with  VEs  (Stanney,  2002).  Recently,  Scerbo 
et  al.  (2005)  used  a  VE  to  examine  emergency  surgical 
procedures  under  hazardous  conditions.  They  argued  that 
VEs  can  be  a  valuable  tool  for  medical  training  because 
they  provide  a  rich  context  in  which  to  examine 
performance.  The  benefits  of  this  approach  are 
numerous.  First,  VEs  provide  a  safe  environment  for 
training  medical  personnel  under  a  variety  of  stressful 
conditions.  Second,  VEs  extend  the  range  of  applications 
for  current  medical  simulators.  Thus,  a  VR  medical 
simulator  can  be  embedded  in  a  VE  to  examine 
contextual  factors  on  performance.  Most  important,  VEs 
allow  one  to  examine  performance  without  putting  any 
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patients  at  risk.  In  this  regard,  they  offer  a  laboratory  in 
which  to  study  new  training  techniques. 

4)  Team  training 

Many  medical  VR  simulators  address  surgical 
procedures.  Although  these  simulators  are  typically 
designed  to  train  individual  skills,  surgical  procedures 
are  actually  performed  in  a  team  context  that  includes 
the  surgeon,  anesthesiologist,  surgical  and  anesthetic 
nurses,  possibly  trainees  (e.g.,  residents,  medical 
students,  etc.),  and,  of  course,  the  patient.  Many  human 
factors  specialists  have  experience  with  training 
paradigms  that  address  both  individuals  and  teams 
(Swezey  &  Salas,  1992).  Further,  principles  of  crew 
resource  management  that  have  been  effective  in  the 
aviation  community  are  now  being  applied  to  medical 
teams  (Gaba,  1994;  Helmreich  &  Schaefer,  1994); 
however,  the  emphasis  thus  far  has  been  aimed  primarily 
at  anesthesiologists.  This  team  perspective  has  not  been 
wholly  embraced  by  those  who  study  VR-based 
simulators.  Thus,  there  is  a  need  to  expand  the  study  of 
medical  team  performance  to  include  a  wider  range  of 
simulators  and  address  the  unique  needs  of  each  team 
member. 

5)  Errors 

In  the  recent  Institute  of  Medicine  report.  To  Err  is 
Human  (Kohn,  Corrigan,  M.,  &  Donaldson,  1999), 
medical  errors  were  estimated  to  contribute  to  as  many 
as  98,000  deaths  annually  in  U.S.  hospitals.  The  report 
has  had  a  significant  impact  in  the  medical  profession  as 
health  care  providers  and  the  institutions  that  train  them 
attempt  to  understand  and  address  this  problem. 

Although  medical  simulators  are  being  promoted  as  a 
partial  solution  to  reducing  medical  errors,  it  is  not  clear 
that  developers  are  creating  simulators  from  a 
fundamental  understanding  of  human  error. 

The  study  of  human  error  is  a  cornerstone  of  human 
factors.  Models  of  human  error,  in  general,  have  been 
offered  by  Reason  (1990),  Moray  and  Senders  (1991), 
and  Wickens  (1992).  Moreover,  Bogner  (1994;  2004) 
and  others  have  been  studying  human  error  in  medicine 
for  over  a  decade.  Further,  human  factors  professionals 
are  also  knowledgeable  about  techniques  for 
investigating  human  error  and  accidents  (Strauch,  2002; 
Wiegmann,  &  Shappell,  2003).  Clearly,  human  factors 
has  much  to  offer  in  the  area  of  medical  error. 

6)  Workload,  stress,  and  fatigue 

Surgeons,  and  particularly  residents,  are  required  to 
work  very  long  hours,  under  moderate  to  severe  levels  of 


sleep  deprivation,  simultaneously  manage  multiple 
patient  cases  that  vary  in  condition  and  severity,  work  in 
teams  whose  members  change  on  a  continual  basis, 
while  literally  holding  the  lives  of  their  patients  in  their 
hands.  In  fact,  surgery  is  rarely  practiced  in  a  low 
workload  environment  free  from  stress  and  fatigue. 
However,  there  has  been  little  systematic  research  on  the 
effects  of  workload,  stress,  and  fatigue  on  surgical 
performance  because  efforts  to  study  these  factors  may 
exacerbate  their  effects  and  put  patients  at  greater  risk. 

Medical  simulators  offer  a  reasonable  compromise. 
They  provide  an  opportunity  to  manipulate  and  study 
workload,  stress  and  fatigue  in  a  safe,  controlled 
environment.  Human  factors  researchers  have  a  long 
history  of  studying  workload,  stress,  and  fatigue  as  well 
as  designing  countermeasures  to  their  deleterious  effects 
(e.g.,  Hancock  &  Desmond,  2001)  and  could  provide 
invaluable  help  addressing  these  issues  in  surgery. 

7)  Haptic  and  multimodal  perception 

At  present,  the  models  underlying  the  haptic 
feedback  systems  in  most  medical  VR  simulators  are 
still  crude.  The  ability  to  faithfully  reproduce  the 
response  of  soft  tissue  continues  to  present  a  significant 
design  challenge.  Moreover,  some  developers  and  even 
physicians  have  questioned  the  need  for  high  fidelity 
haptics  or  any  haptics  at  all  (Montgomery,  2005).  There 
are  many  researchers  within  our  profession  who 
understand  haptic  perception  and  the  psychophysical 
methods  for  comparing  and  equating  real  and  virtual 
objects  in  haptic  space  (Dow,  Thomas,  &  Johnson,  1999; 
Biggs,  &  Srinivasan,  2002).  We  can  help  develop  haptic 
response  systems  that  feel  natural.  Further,  other  human 
factors  researchers  study  multimodal  perception  and 
understand  the  challenges  associated  with  designing 
multimodal  VR  systems  that  include  visual,  auditory, 
and  haptic  displays  (Nelson  &  Bolia,  2002). 

8)  Cognitive  engineering 

Most  current  medical  VR  systems  focus  on  the 
psychomotor  skills  needed  to  perform  specific 
procedures.  Although  the  early  years  of  a  physician’s 
residency  are  aimed  at  developing  these  skills,  in  later 
years  physicians  spend  more  of  their  time  addressing  the 
cognitive  skills  required  for  diagnoses,  decision-making, 
problem-solving,  and  management.  Thus,  there  is  need 
to  include  these  aspects  of  a  physician’s  job  in  future 
simulators  (Jarrell,  2005).  Again,  many  human  factors 
researchers  and  designers  have  extensive  experience  in 
cognitive  task  analysis,  cognitive  engineering,  intelligent 
interface  design,  and  automated  systems.  The  designers 
of  medical  VR  systems  will  need  to  draw  upon  this 
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knowledge  in  order  to  create  the  next  generation  of 
simulators  that  benefit  more  advanced  residents  or 
practicing  physicians. 

9)  Cost  justification 

Human  factors  professionals  have  long  understood 
the  economic  benefits  that  come  from  well-designed 
products  (Alexander,  1994).  Hendrick  (1996)  described 
numerous  examples  of  ergonomic  solutions  that  paid  big 
dividends  in  cost  reductions  or  increased  sales.  Others 
have  described  methods  for  using  cost-benefit  analysis 
to  show  the  value  of  human  factors  contributions  within 
the  product  development  life  cycle  (see  Bias  &  Mayhew, 
1994).  Those  who  work  more  closely  with  simulation- 
based  training  systems  have  described  methods  for 
determining  the  cost  effectiveness  of  such  systems 
(Farmer,  van  Rooij,  Riemersma,  Joma,  &  Moraal,  1999). 

The  medical  simulation  community  still  has  not 
given  much  consideration  to  the  economic  impact  of 
their  systems.  Although  Satava  (2001)  has  discussed  the 
expenses  associated  with  developing  medical  simulators, 
few  have  attempted  to  measure  the  cost  savings.  Many 
practitioners  within  the  human  factors  community 
recognize  that  calculating  the  added  value  of  ergonomic 
solutions  is  not  only  a  necessary  component  of 
feasibility  studies  for  individual  projects,  but  important 
for  the  discipline  as  a  whole.  The  medical  simulation 
community  would  be  well  served  to  follow  the  ground 
work  laid  by  human  factors  professionals  where  cost 
justification  is  concerned. 

10)  Validity,  control,  and  experimental  design 

There  is  great  interest  among  the  developers  and 
users  of  medical  VR  simulators  to  show  that  they  are 
valid  and  that  training  on  a  simulator  transfers  to 
procedures  performed  on  genuine  patients. 

Unfortunately,  much  of  the  research  conducted  to 
support  these  efforts  suffers  from  a  variety  of 
methodological  weaknesses.  The  research  literature  is 
often  plagued  with  simplistic  or  even  incorrect  notions 
of  construct,  internal,  and  external  validity.  For  instance, 
it  is  not  uncommon  for  investigators  to  claim  they  have 
established  construct  validity  without  ever  mentioning 
the  underlying  construct.  Some  of  research  also  suffers 
from  inadequate  controls  or  specification  of  control 
procedures.  Often,  good  experimental  control  is 
exercised,  but  for  the  least  likely  threats  to  validity, 
while  the  most  serious  threats  go  uncontrolled. 

Most  human  factors  specialists  have  strong 
methodological  and  quantitative  skills  and  can  design 
experiments  that  minimize  threats  to  internal  validity 
while  maximizing  opportunities  for  external  validity. 


More  important,  human  factors  researchers  often  design 
studies  in  which  sources  of  variance  can  be  identified 
and  the  mechanisms  underlying  results  obviated. 

CONCLUSION 

Obviously,  there  are  other  human  factors  issues  that 
could  be  added  to  this  list.  But,  there  are  also 
technological  challenges  that  are  equally  significant. 
Some  of  these  include  better  physical  models  of  tissues 
and  organs,  more  extensive  representation  of  disease  and 
pathology,  establishment  of  interoperability  standards 
among  systems,  and  how  best  to  integrate  simulators 
into  current  medical  curricula. 

Clearly,  medical  VR  simulators  have  arrived  and 
are  already  having  a  profound  impact  on  the  way 
medicine  is  being  taught.  For  the  first  time  in  the  history 
of  medicine,  they  offer  the  opportunity  for  the  objective 
assessment  of  performance  with  standardized  metrics  in 
a  safe  and  controlled  manner.  They  will  increase  skill 
proficiency  and  reduce  the  performance  variance  and 
errors  of  those  who  practice  medicine  and  ultimately, 
will  provide  new  and  verifiable  ways  to  improve  patient 
safety.  Gallagher  (2004)  has  suggested  that  in 
evolutionary  terms,  current  VR  simulators  in  medicine 
are  on  a  par  with  the  Link  flight  simulator  introduced  in 
the  1930s.  We  in  the  human  factors  community 
recognize  the  innumerable  and  significant  contributions 
we  have  made  to  the  evolution  of  aviation  in  the  areas  of 
design,  training,  and  safety  in  large  part  due  to 
simulation  technology.  Perhaps,  70  years  from  now  we 
will  be  able  to  look  back  on  our  role  in  medicine  and 
healthcare  and  recognize  the  innumerable  and  significant 
contributions  we  made  through  our  involvement  in  the 
evolution  of  medical  simulators. 
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ABSTRACT 

UNDERSTANDING  THE  NATURE  OF  SURGICAL  EXCELLENCE  USING 
A  COMPETENCY  MODELING  APPROACH 

Hope  S.  Hanner-Bailey 
Old  Dominion  University,  2006 
Director:  Dr.  Mark  W.  Scerbo 

Currently,  a  thorough  description  of  the  knowledge,  skills,  abilities,  and  other  personal 
characteristics  (KSAOs)  that  distinguish  an  exceptional  surgeon  does  not  exist.  This 
knowledge  is  pertinent  to  the  development  of  selection,  training,  and  performance 
assessment  methods  that  can  be  used  to  develop  high  performing  surgeons.  Expert 
surgeons  from  around  the  country  were  recruited  to  participate  in  an  interview  to  discuss 
the  KSAOs  (i.e.,  the  competencies)  needed  to  be  exceptional  in  the  field.  A  smaller 
number  of  novice  surgeons  were  also  interviewed.  The  expert  interview  data  were 
distilled  into  a  competency  model  that  consists  of  ten  competencies  and  patterns  within 
the  data  that  distinguished  between  the  expert  and  novice  results  were  examined.  Results 
revealed  ten  competencies  essential  for  distinguishing  between  outstanding  and  typical 
and  surgeons:  Dedication  to  Patient  Care,  Integrity,  Tireless  Work  Ethic,  Preparedness, 
Intellectual  Giftedness  and  Curiosity,  Humility,  Compassion,  Devotion  to  Field,  Rapid 
Decision  Making,  and  Passion  for  Teaching.  These  results  suggest  that  surgical  training 
would  benefit  from  focusing  on  the  development  and  enhancement  of  character-based 
competencies.  Once  validated,  Industrial-Organizational  psychologists  can  use  the 
present  model  to  assist  surgeons  in  creating  instruments  that  measure  and  track  residents’ 
acquisition  of  these  competencies. 
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INTRODUCTION 

During  the  Annual  Meeting  of  the  Society  of  University  Surgeons  in  2003,  Dr. 
William  Silen  of  Harvard  Medical  School  delivered  the  keynote  address  in  which  he 
described  the  current  state  of  surgical  education.  Silen  argued  that  a  major  change  was 
needed  in  surgical  residency  programs.  In  this  address,  he  shared  the  following  statement 
made  by  a  surgical  resident  from  a  reputable  medical  school: 

I  believe  the  apprenticeship  model,  so  successful  in  the  mid  portion  of  the  20th 
century,  is  now  dead  and  has  taken  the  spirit  out  of  the  work  of  medicine.  There 
are  too  many  sick  patients  to  be  cared  for  by  too  few  people... Time  spent 
practicing,  teaching,  thinking  are  considered  luxuries,  taken  with  the  knowledge 
that  each  minute  will  subtract  from  time  [treating  patients]  or  from  time  sleeping, 
(p.  399) 

Others  have  recently  expressed  similar  attitudes  regarding  this  shift  in  surgical  education. 
For  example,  Gawande  (2001)  stated  that  residency  programs  must  acknowledge  that 
although  the  method  of  leaming-by-doing  (i.e.,  on-the-job  training)  worked  for  decades, 
its  flaws  are  increasingly  evident.  Silen  and  Gawande  represent  a  growing  contingent  of 
educators  who  are  endorsing  a  major  modification  to  surgical  training. 

Such  remarks  also  indicate  that  many  surgical  programs  have  reached  a 
crossroads.  The  Halstedian  method,  the  reigning  teaching  strategy  for  the  past  century, 
consists  of  a  “see  one,  do  one,  teach  one”  approach  to  surgical  training.  In  other  words, 
the  Halstedian-based  system  to  training  surgical  residents  mirrors  a  traditional 
apprenticeship  model.  This  method  was  a  logical  approach  to  surgery  when  there  was  no 
viable  alternative  to  learning  directly  with  patients.  The  consensus  in  surgical  education 
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is  that  the  Halstedian  method  is  now  insufficient  given  the  changing  competencies 
required  for  surgeons  (Fabri,  2003;  Hall,  2004;  Haluck  &  Krummel,  2000).  The  concern 
over  this  teaching  method  is  so  widespread  that  formalized  training  programs  of  basic 
surgical  skills  have  been  established  in  many  countries,  including  the  United  Kingdom 
and  Australia  (Davies  &  Hamdorf,  2003).  A  good  example  of  such  a  skills  training 
facility  is  The  Centre  for  Medical  and  Surgical  Skills  (CTEC)  in  Perth,  Western 
Australia,  a  state-of-the-art  training  center  consisting  of  a  medical  and  surgical  workshop, 
a  high-fidelity  simulated  anesthesia  mannequin,  and  a  virtual  hospital  that  contains  an 
operating  theater,  emergency  department,  and  intensive  care  unit. 

Because  few  surgical  programs  are  equipped  with  such  technically  advanced 
training  centers,  most  education  continues  to  occur  in  the  operating  room  (OR).  From  an 
educational  standpoint,  the  OR  is  a  poor  classroom  for  learning  important  surgical  skills. 
It  is  an  uncomfortable,  stressful,  and  even  hostile  environment;  thus,  learning  potential 
and  the  ability  for  a  surgeon  to  concentrate  are  often  compromised  (Haluck  &  Krummel, 
2000).  Although  many  residency  programs  need  to  improve  aspects  of  their  current 
curricula,  it  is  of  even  greater  importance  to  determine  whether  these  programs  are 
producing  competent  surgeons.  To  do  so,  the  surgical  community  must  first  reach  a 
consensus  regarding  the  definition  of  a  competent  surgeon.  Currently,  surgical 
competency  is  an  elusive  concept  that  even  the  American  Board  of  Surgery  is  grappling 
to  define  (Scott  et  al.,  2000).  However,  this  Board  does  suggest  that  multiple  assessment 
instruments  be  used  to  evaluate  competency.  This  recommendation  implies  that  surgical 
competency  is  multidimensional  in  nature  and  therefore,  assessing  only  medical 
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knowledge  or  technical  skill  does  not  provide  a  complete  picture  of  a  surgeon’s 
proficiency. 

Of  equal  significance  to  the  concept  of  surgical  competency  is  the  role  of 
assessment.  It  is  important  to  investigate  whether  educators  can  adequately  assess  the 
levels  of  competency  that  surgical  residents  achieve  throughout  their  training.  The 
identification  of  poor  performers  is  important  because  these  individuals  require  additional 
training  or,  in  extreme  cases,  dismissal  from  the  surgical  program  altogether. 
Unfortunately,  existing  assessment  methods  are  often  criticized  as  subjective, 
inconsistent,  and  too  general  to  discriminate  among  trainees  (Cheung  &  Yau,  2002; 
Gibbons,  Baker,  &  Skinner,  1986;  Paisley,  Baldwin,  &  Paterson-Brown,  2001). 

Assessing  Surgical  Performance 

Little  has  been  written  about  characteristics  that  distinguish  a  typical  surgeon 
from  an  exceptional  one.  However,  all  surgeons,  even  the  exceptional  ones,  begin  their 
training  in  medical  school.  Upon  graduating  from  college,  aspiring  physicians  must  first 
attend  medical  school,  where  they  acquire  a  broad  education  in  medicine  typically  within 
a  four-year  program.  The  first  two  years  are  taught  largely  via  classroom  lectures, 
laboratories,  and  demonstrations.  The  object  of  these  two  years  is  to  provide  students 
with  a  foundation  in  medical-related  sciences.  During  the  third  and  fourth  years,  students 
complete  various  medical  rotations  in  some  of  the  main  specialties  of  medicine  such  as 
obstetrics/gynecology,  family  medicine,  psychiatry,  and  general  surgery.  Once  the  four 
years  are  completed,  students  receive  their  medical  degrees  and  are  then  matched  to 
residency  programs  within  their  chosen  specialty.  These  doctors  are  now  referred  to  as 
residents.  In  essence,  residency  is  a  time  when  surgeons  are  expected  to  sharpen  their 
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skills  and  further  their  medical  knowledge.  The  first  year  of  residency  is  known  as  the 
internship,  although  the  internship  can  be  as  long  as  three  years  in  certain  institutions 
(Ludmerer,  1999).  Once  residency  is  concluded,  the  formal  portion  of  a  surgeon’s 
medical  education  is  considered  to  be  complete.  But  the  question  remains,  how  good  is  a 
surgeon  at  what  he  or  she  does? 

Until  recently,  surgical  performance  was  defined  by  the  raw  number  of  medical 
procedures  performed  and  patients  examined  (Haluck  &  Krummel,  2000).  Due  to  a  recent 
shift  in  educational  standards  and  tremendous  advances  in  medical  technology  (e.g., 
medical  simulators),  this  approach  is  becoming  obsolete.  Rather  than  defining  level  of 
performance  by  a  set  number  of  procedures,  many  surgical  programs  are  turning  to 
predetermined  educational  goals  and  objectives  for  assessment  purposes  (Romanchuk, 
2004). 

The  need  to  operationalize  surgical  performance  is  widespread.  In  fact,  there  has 
been  extensive  public  interest  and  scrutiny  in  defining  levels  of  surgical  performance 
(Satava,  Gallagher,  &  Pellegrini,  2003).  The  media,  patient  groups,  governmental 
agencies,  and  insurance  companies  are  a  few  of  the  many  parties  involved  in  this  debate 
(Singh,  Smeeton,  &  O’Brien,  2003).  Most  medical  professionals  believe  that  a  surgeon’s 
level  of  performance  can  be  determined  by  considering  an  amalgam  of  several  skills  and 
abilities  including,  clinical  skills,  technical  ability,  academic  potential,  administrative 
skills,  good  judgment,  and  the  ability  to  maintain  strong  relationships  with  patients  and 
other  health  care  professionals  (Cuschieri,  Francis,  Crosby,  &  Hanna,  2001).  Patil, 

Cheng,  and  Wong  (2003)  agree  that  a  high  performing  surgeon  must  have  strong  clinical 
skills,  a  compassionate  and  professional  attitude,  and  effective  communication  skills,  but 
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they  also  state  that  a  high  performing  surgeon  must  possess  the  ability  to  think  and  work 
in  highly  stressful  situations. 

Hall,  Ellis,  and  Hamdorf  (2002)  acknowledge  that  although  technical  ability  is 
crucial,  a  high  performing  surgeon  must  also  be  a  capable  decision  maker.  It  has  been 
estimated  that  a  surgical  procedure  requires  approximately  75  percent  cognitive  skills  and 
25  percent  technical  skills  (Satava  et  al.,  2003;  Spencer,  1978).  Surgical  performance,  by 
its  very  nature,  is  complex  because  it  requires  good  manual  dexterity,  a  blend  of  high 
stakes  decision  making,  and  critical  thinking  in  complex  situations  often  with  relatively 
uncommon  procedures  (Hall,  2004;  Higgins  et  al.,  2004;  Spanknebel  et  al.,  2004).  There 
have  been  many  attempts  to  define  surgical  competency,  each  with  its  own  merits  and 
weaknesses. 

Historical  Attempts  to  Assess  Surgical  Performance 

Mortality  Rates 

One  fairly  straightforward  approach  to  determining  the  quality  of  a  surgeon’s 
level  of  performance  is  simply,  the  death  of  a  patient  (e.g.,  Margulies  et  al.,  2001;  Singh, 
Smeeton,  &  O’Brien,  2003).  Mortality  rates  allow  hospital  administrators  and  patients  to 
determine  the  quality  of  surgical  care  they  are  likely  to  receive.  In  a  recent  study  aimed  at 
identifying  under-performing  surgeons,  Singh  et  al.  (2003)  provided  an  example  of 
operationalizing  surgical  performance.  They  used  a  mortality  rate  of  four  percent  over  a 
30-day  period  following  a  radical  cystectomy  procedure  for  bladder  cancer.  Acceptable 
surgical  performance  was  defined  as  a  mortality  rate  of  four  percent  and  under¬ 
performance  was  defined  as  a  mortality  rate  of  eight  percent  or  greater. 
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Although  mortality  rates  are  clinically  relevant  and  certainly  important  to  patient 
groups  and  insurance  companies,  there  are  problems  inherent  with  this  approach.  For 
instance,  very  large  sample  sizes  are  needed  which  often  create  logistical  problems  in 
medical  research.  Furthermore,  since  surgery  is  a  team  effort,  a  surgeon  is  rarely  the  sole 
person  responsible  for  the  death  of  a  patient.  If  a  surgical  team  fails  to  communicate  or 
provide  proper  monitoring  and  feedback  during  a  procedure,  it  may  not  be  possible  to 
place  blame  on  any  particular  individual.  As  such,  defining  surgical  performance  through 
mortality  statistics  may  be  viewed  as  both  a  contaminated  and  deficient  measure  of 
performance.  It  may  be  contaminated  because  the  surgeon  may  not  necessarily  be 
responsible  for  problems  that  arose  at  the  team  level.  Mortality  rates  may  also  be 
deficient  because  they  do  not  provide  any  evaluative  information  regarding  other 
important  characteristics  of  job  performance,  such  as  interpersonal  interactions  with 
patients  and  their  family  members. 

Patient  Satisfaction 

The  quality  of  a  surgeon’s  performance  is  not  only  tied  to  patient  survival,  but 
also  the  attitudes  and  reactions  held  by  their  patients.  Although  medical  practitioners  are 
evaluated  by  their  superiors,  many  hospitals  and  clinics  now  include  their  patients  in  the 
evaluation  process.  Over  the  past  25  years,  patients’  opinions  have  become  a  serious 
concern  in  the  medical  and  nursing  literature  (Sitzia  &  Wood,  1997).  For  example, 
patient  satisfaction  with  physicians’  interpersonal  skills  has  been  shown  to  be  related  to 
long-term  psychological  adjustment.  Mager  and  Andrykowski  (2002)  interviewed  60 
women  diagnosed  with  local  or  regional  breast  cancer  to  examine  the  relationship 
between  breast  cancer  survivors'  experiences  during  their  diagnostic  consultation  and 
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their  subsequent  psychological  adjustment.  One  interesting  finding  was  that  the  best 
predictor  of  long-term  patient  depression,  stress,  and  anxiety  was  not  perceptions  of 
physician  competence,  but  patients’  perceptions  of  a  physician’s  interpersonal  skills. 
Thus,  patients  may  require  more  than  technical  expertise  from  their  physicians;  patients 
typically  expect  to  be  treated  with  sensitivity  and  respect. 

Assuming  that  patients  are  well-informed  and  well-intentioned,  the  information 
gathered  from  patient  satisfaction  questionnaires  can  be  extremely  useful.  On  the  other 
hand,  some  physicians  are  suspicious  of  patient  satisfaction  data.  They  believe  that  these 
data  may  unduly  threaten  the  status  and  livelihood  of  medical  practitioners,  as  well  as 
threaten  the  professional  standards  of  the  medical  community  as  a  whole  (Armstrong, 
1991).  Although  such  physicians  may  assume  that  many  patients  are  dissatisfied  with 
their  medical  care,  in  practice  the  opposite  is  usually  true  (Sitzia  &  Wood,  1997).  Since 
these  fears  are  likely  to  be  unfounded,  patient  satisfaction  information  could  be  an 
important  and  worthwhile  component  of  surgical  excellence. 

To  date,  there  are  many  more  studies  involving  patient  satisfaction  with  non- 
surgical  physicians  than  there  are  with  surgeons.  However,  at  the  University  of  Toronto, 
MacRae,  Cohen,  Regehr,  Reznick,  and  Burstein  (1997)  developed  a  six-station,  three- 
hour,  standardized-patient-based  evaluation  system.  MacRae  et  al.  called  this  system  the 
Patient  Assessment  and  Management  Examination  (PAME).  They  used  it  to  assess  18 
senior  general  surgery  residents.  Although  the  sample  size  was  small,  their  results 
suggested  that  the  PAME  is  a  reliable  and  useful  tool  for  assessing  the  attitudes  of 
surgical  patients.  However,  it  must  also  be  acknowledged  that  most  patients  have  limited 
insight  into  an  operative  procedure.  Typically,  patients  are  able  to  judge  only 
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interpersonal  interactions  with  the  surgeon  as  well  as  the  final  outcome  of  a  procedure. 
While  patient  input  provides  an  important  measure  of  a  surgeon’s  performance,  patients 
are  not  capable  of  providing  a  complete  evaluation  of  surgical  ability. 

Suturing  Skill 

Another  approach  that  has  been  used  to  define  surgical  performance  is  the 
assessment  of  suturing  skills.  Those  who  use  suturing  as  a  measure  of  surgical 
performance  seem  drawn  to  its  objectivity.  In  these  studies,  a  standardized  battery  of 
knot-tying  and  suturing  stations  may  be  used  to  assess  performance  (Bann,  Khan,  Datta, 

&  Dazi,  2004).  Although  suturing  is  not  the  only  skill  necessary  for  technical 
performance,  it  is  a  fundamental  aspect  of  surgery.  Seki  (1987,  1988)  conducted  several 
studies  dealing  with  the  suturing  techniques  among  surgeons.  His  work  demonstrated 
that  surgeons  may  overestimate  their  accuracy  of  suturing  and  cause  tissue  damage  by 
wavering  the  needle  without  realizing  it.  Furthermore,  Seki’s  research  demonstrated  that 
less  experienced  surgeons  benefited  more  from  training  than  more  experienced  ones. 
Khan,  Bann,  Darzi,  and  Butler  (2003)  asked  a  group  of  43  plastic  surgeons  and  46 
general  surgeons  to  suture  a  wound  made  in  a  latex  pad  using  interrupted  sutures.  They 
were  videotaped  performing  the  procedure  while  their  movements  were  monitored  using 
an  electromagnetic  tracking  system.  All  videotapes  were  assessed  via  the  Objective 
Structured  Assessment  of  Technical  Skills  (OSATS),  which  is  an  operation-specific 
checklist  that  is  frequently  used  to  assess  surgical  skills  among  residents  in  teaching 
hospitals  (Faulkner,  Regehr,  Martin,  &  Reznick,  1996;  Hernandez  et  al.,  2004;  Reznick, 
Regehr,  MacRae,  Martin,  &  McCulloch,  1997).  This  assessment  method  allows  trainees 
to  be  scored  objectively  by  their  peers  or  superiors  (for  further  description  of  the  OSATS, 
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see  the  following  section,  “Technical  Skill”).  Khan  et  al.  found  that,  when  suturing, 
plastic  surgeons  were  significantly  more  efficient  with  their  time  than  general  surgeons. 
They  also  found  that  senior  surgeons  performed  better  than  their  juniors,  regardless  of 
surgical  specialty. 

Collectively,  these  studies  suggest  that  suturing  skill  may  be  useful  as  a  measure 
of  surgical  performance  for  two  reasons.  First,  suturing  assessment  may  be  a  sensitive 
measure  because  it  has  been  shown  to  discriminate  among  surgeons  from  different 
specialties  and  with  different  levels  of  experience.  Second,  suturing  may  provide  an 
index  of  one’s  overall  level  of  technical  competence. 

Technical  Skill 

Psychomotor  abilities  and  technical  skills  have  been  an  important  part  of  selection 
and  assessment  research  for  the  past  80  years  (Ackerman,  Cianciolo,  &  Bowen,  1999). 
Technical  skills  have  played  a  role  in  predicting  individual  differences  in  various  skilled 
professions.  However,  the  convention  in  Industrial-Organizational  psychology  is  to  lump 
technical  skills  into  a  “noncognitive  skills  and  abilities”  category  (Schmidt,  Ones,  & 
Hunter,  1992).  Other  terms  that  fall  under  the  “noncognitive”  umbrella  are  psychomotor 
and  perceptual  ability.  Although  general  cognitive  ability  is  considered  to  be  the 
strongest  predictor  of  job  performance  across  a  wide  range  of  jobs,  noncognitive  abilities 
may  improve  the  overall  predictive  validity  of  cognitive  ability  measures  (e.g.,  Schmidt 
&  Hunter,  1998,  McHenry,  Hough,  Toquam,  Hanson,  &  Ashworth,  1990). 

Approximately  twenty  years  ago,  the  United  States  Army  began  a  seven-year 
research  program  to  improve  the  selection  and  classification  of  entry-level  military 
members.  This  effort,  known  as  Project  A,  was  one  of  the  few  examples  of  empirical 
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research  in  which  the  role  of  noncognitive  abilities  in  selection  was  examined  (McHenry, 
et  al.,  1990).  Results  suggested  that  while  cognitive  ability  was  the  best  predictor  of  job 
performance  across  nine  different  military  occupations,  adding  specific  ability  tests 
increased  the  validity  beyond  what  was  predicted  by  cognitive  ability  alone  (McHenry  et 
al.,  1990).  Johnston  and  Catano  (2002)  also  examined  the  predictive  and  incremental 
validity  of  noncognitive  abilities.  They  administered  tests  of  manual  dexterity,  finger 
dexterity,  and  motor  coordination  to  209  Canadian  Forces  trainees  in  technical  and 
mechanical  occupations.  The  results  indicated  that  only  manual  dexterity  predicted 
training  performance.  Further,  although  the  addition  of  the  three  psychomotor  tests 
increased  validity  beyond  using  only  cognitive  measures,  only  manual  dexterity 
significantly  contributed  to  the  regression  model. 

Logic  suggests  that  technical  skill  is  related  to  surgical  performance.  However, 
until  the  past  decade,  the  formal  assessment  of  surgeons’  technical  skills  was  often 
accomplished  via  only  a  single  item  on  summary  achievement  indicators,  such  as  in¬ 
training  reports  (Winckel,  Reznick,  Cohen,  &  Taylor,  1994).  Until  the  OSATS  was 
introduced  into  the  surgical  community,  the  assessment  of  technical  skills  was  much  less 
sophisticated  than  the  assessment  of  clinical  skills,  cognitive  ability,  and  medical 
knowledge.  The  OSATS  is  comprised  of  six  separate  tasks  that  a  trainee  must  perform 
within  90  minutes.  These  tasks  include  excision  of  a  skin  lesion,  insertion  of  a  T-tube, 
abdominal  wall  closure,  control  of  major  bleeding,  stapled  bowel  anastomosis,  and 
sutured  bowel  anastomosis  (Faulkner,  Regehr,  Martin,  &  Reznick,  1996).  Using  a  task- 
specific  checklist  and  a  global  rating  scale,  the  examiner  is  able  to  assess  the  trainee’s 
performance. 
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Technical  skill  may  also  be  assessed  more  generally  using  human  cadavers, 
animal  models,  and  living  patients.  There  are  obvious  ethical  constraints  to  using  living 
patients  since  safety  must  always  precede  training  opportunities.  These  constraints  may 
also  hinder  the  assessment  process,  particularly  when  an  examiner  must  step  in  to  assist  a 
trainee  if  something  goes  wrong  during  the  procedure.  Such  external  help  or  guidance 
would  bias  the  evaluation  process.  Cadavers  therefore  alleviate  some  of  the  ethical 
concerns  and  problems  related  to  assessment.  However,  since  tissue  supply  is  limited  in 
cadavers  and  tissue  properties  are  different  from  those  of  live  patients,  the  use  of 
cadavers  also  presents  problems  (Khan,  Bann,  Darzi,  &  Butler,  2003).  Furthermore, 
human  cadavers  may  place  trainees  at  risk  because  it  is  possible  to  transfer  disease  from 
cadavers  to  humans. 

Recently,  surgical  educators  have  taken  advantage  of  computer-enhanced 
technology  to  train  residents  to  perform  specific  procedures,  as  well  as  to  improve  their 
overall  technical  skills.  Several  virtual  reality  (VR)  simulators  that  address  endoscopy 
and  laparoscopy  are  now  available  to  train  instrument  manipulation  and  advanced 
procedures  (Patil,  Cheng,  &  Wong,  2003).  As  surgery  becomes  less  invasive,  the  range 
of  skills  required  to  manage  surgical  issues  is  increasing.  Compared  to  traditional  open 
procedurs,  minimally  invasive  surgery  (MIS)  involves  smaller  incisions  and  less 
extensive  manipulation  of  the  tissues  surrounding  the  target  structure  (Guyer,  Foley, 
Phillips,  &  Ball,  2003).  Further,  patients  who  undergo  minimally  invasive  procedures 
typically  experience  less  pain,  a  smoother  recovery,  and  an  improved  cosmetic  outcome 
(Cooley,  1998). 
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Researchers  acknowledge  that  training  and  assessment  of  the  skills  needed  for 
MIS  differ  somewhat  from  the  skills  needed  for  conventional  surgical  procedures  (e.g., 
Seymour  et  al,  2002).  The  Minimally  Invasive  Surgical  Trainer- Virtual  Reality  (MIST®- 
VR)  has  become  one  of  the  most  popular  trainers  and  assessment  systems  of  the  technical 
skills  needed  to  perform  MIS  (Jordan,  Gallagher,  McGuigan,  McGlade,  &  McClure, 
2000).  This  PC-based  system,  which  contains  two  laparoscopic  instruments  and  a 
diathermy  pedal,  has  six  tasks  of  increasing  difficulty.  The  tasks  use  abstract  graphics 
that  simulate  the  manipulations  required  during  laparoscopic  cholecystectomy 
(Grantcharov,  Bardram,  Funch-Jensen,  &  Rosenberg,  2001).  The  MIST®-VR  system 
records  number  of  errors,  economy  of  movement  for  each  hand  (actual  path  length/ideal 
path  length),  and  time  information.  This  scoring  system  permits  measurement  of  skills 
relevant  to  laparoscopic  surgery.  Moreover,  research  has  shown  that  individuals  trained 
on  the  MIST®-VR  demonstrate  significantly  better  technical  performance  than  those 
surgeons  trained  using  traditional  methods  (Seymour  et  al.,  2002). 

As  with  any  assessment  method,  one  must  consider  its  advantages  and 
disadvantages.  One  obvious  benefit  of  defining  surgical  ability  by  technical  skills  is  its 
intuitive  appeal.  Instrument  handling  and  the  ability  to  demonstrate  respect  for  tissue  are 
necessary  prerequisites  for  any  surgical  procedure.  However,  the  question  then  begs,  is  a 
surgeon  truly  exceptional  if  he  or  she  is  technically  proficient  but  has  poor  interpersonal 
and  communication  skills  and  is  lacking  in  overall  professionalism?  Although  technical 
skill  is  important  and  clearly  necessary  to  perform  surgical  procedures,  it  does  not 
provide  information  about  other  aspects  of  surgery  that  are  relevant  to  patient  care.  Thus, 
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for  the  present  study,  it  is  anticipated  that  technical  skill  will  not  emerge  as  a  competency 
that  distinguishes  typical  from  exceptional  surgeons. 

Critical  Thinking 

Surgery  consists  of  activities  that  are  highly  complex  and  risky.  In  fact,  much 
medical  decision  making  is  based  on  insufficient  information  (Satish  et  al.,  2001).  As 
such,  a  surgeon  often  needs  to  make  life  or  death  decisions  during  specialized,  intricate, 
and  sometimes,  relatively  uncommon  procedures  (Spanknebel,  Shoup,  Temple,  Coit, 
Brennan,  &  Jaques,  2004).  The  majority  of  surgical  research  focuses  on  thought 
processes  involving  major  decisions  such  as  deciding  whether  to  operate,  determining 
diagnoses,  and  deciding  the  level  of  risk  to  which  a  patient  can  be  exposed  by  operating 
(Birkmeyer  &  Birkmeyer,  1996).  With  the  advent  of  MIS,  surgeons  must  now  make 
important  mid-surgery  decisions  such  as  whether  to  convert  from  a  minimally  invasive 
procedure  to  an  open-incision  procedure  (Dominguez,  2001). 

To  be  a  high  performing  surgeon,  it  is  likely  that  one  needs  to  possess  a 
combination  of  technical  skills  and  higher-order  cognitive  processes,  such  as  critical 
thinking  and  independent  thought.  Although  surgeons  must  be  intellectually  prepared  to 
handle  a  wide  array  of  contingencies,  many  training  programs  are  not  designed  to 
produce  surgeons  who  think  critically,  learn  independently,  and  generate  new  ideas 
(Ludmerer,  1999).  Curricula  are  often  typified  by  rote  memorization,  lectures,  and  other 
rigid  processes.  In  addition,  the  Halstedian  method  is  essentially  a  behavioral  approach 
to  instruction,  as  observed  behavior  rather  than  internal  processes  is  emphasized  (Hall, 
Ellis,  &  Hamdorf,  2002).  Given  that  the  Halstedian  method  is  becoming  obsolete,  the 
role  of  critical  thinking  is  taking  a  more  prominent  role  in  surgical  programs. 
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Critical  thinking  is  especially  necessary  when  dealing  with  complex  medical  cases 
and  individualized  treatment  plans.  Satish  et  al.  (2001)  also  believe  that  high  performing 
surgeons  require  more  than  strong  technical  skills.  They  contend  that  these  surgeons 
should  have  the  capacity  to  respond  appropriately  to  multiple,  simultaneous  symptoms 
and  medical  events  that  can  potentially  compromise  a  patient’s  safety.  Educating 
surgeons  about  diseases  and  medication  is  fairly  straightforward  information  that  can  be 
translated  through  training,  lectures,  and  medical  textbooks.  However,  as  Satish  et  al. 
(2001)  emphasize,  it  is  the  integration  of  that  information  toward  individualized 
treatment  plans  and  the  response  to  unpredictable  changes  in  the  patient’s  status  that  are 
much  more  difficult  to  convey  through  traditional  skill  training.  Therefore,  these 
researchers  used  an  alternative  assessment  method  to  measure  critical  thinking,  as  well  as 
other  skills  needed  for  integrative  surgical  decision  making  (e.g.,  flexibility,  crisis 
management).  This  method,  known  as  the  Strategic  Management  Simulation  (SMS), 
uses  different  medical  scenarios  to  measure  critical  thinking  and  to  identify  where  an 
individual  needs  improvement.  Satish  and  his  colleagues  used  the  SMS  to  predict 
surgical  residents’  higher  cognitive  function  (e.g.,  critical  thinking).  Their  SMS  results 
were  very  similar  to  ratings  provided  by  surgical  faculty  who  had  known  the  residents  for 
a  minimum  of  two  years. 

Given  how  important  it  is  for  surgeons  to  be  able  to  critically  assess  complex 
medical  cases  and  procedures,  it  is  anticipated  that  critically  thinking  will  emerge  as  a 
competency  in  the  present  study. 
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The  Need  for  Clear  Standards  of  Surgical  Performance 

The  previous  section  described  the  difficulties  researchers  face  when  trying  to 
evaluate  a  surgeon’s  performance.  Nonetheless,  it  is  becoming  more  important  for 
surgeons  to  demonstrate  that  they  can  perform  according  to  professional  standards. 

Recertification  and  Professional  Development 

Professional  organizations  commonly  endorse  certification  and 
recertification/revalidation  programs  for  various  surgical  specialties.  Such  bodies 
prescribe  high  standards  and  requirements  for  both  board  certification  and  recertification. 
For  instance,  before  the  American  Board  of  Colon  and  Rectal  Surgery  (ABCRS)  permits 
a  surgeon  to  apply  for  recertification,  he  or  she  is  required  to:  submit  a  practice  review  of 
consecutive  operative  cases  for  the  preceding  year  and  acquire  1 00  approved  hours  of 
postgraduate  medical  education  in  surgical  or  colorectal  surgery  in  the  two  years  prior  to 
application.  The  purpose  of  these  boards  is  to  ensure  that  surgeons  maintain  acceptable 
qualifications  of  practice  in  their  surgical  specialty,  as  well  as  maintain  commitment  to 
ongoing  education  and  evaluation  of  performance  in  practice  (Patil,  Cheng,  &  Wong, 
2003). 

Although  continuous  medical  education  (CME)  and  recertification  appear  to  be 
universal  endeavors  for  most  surgical  specialties,  ongoing  professional  development 
opportunities  for  those  practicing  general  surgery  is  less  straightforward.  Since  general 
surgery  is  heterogeneous  in  nature,  it  is  much  more  difficult  to  create  surgical 
competence  assessment  sections  that  are  broad  enough  to  encompass  the  spectrum  of 
surgical  circumstances  (Patil  et  al.,  2003).  As  such,  a  major  challenge  facing  the  surgical 
profession  is  to  develop  and  validate  measures  that  assess  a  representative  sample  of 
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general  surgeries.  Moreover,  the  profession  needs  to  create  requirements  that  are  broad 
enough  to  allow  general  surgeons  to  qualify  for  recertification.  Having  discussed 
recertification,  the  resident  work  hour  mandate  is  another  important  issue  facing  surgeons 
today. 

Resident  Work  Hour  Mandate 

Whether  a  resident  chooses  general  surgery  or  some  surgical  specialty,  he  or  she 
is  now  subject  to  limits  in  working  hours  and  training  time.  The  Accreditation  Council 
for  Graduate  Medical  Education  (ACGME),  which  is  the  umbrella  organization  for  all 
residency  review  committees  in  medicine,  promulgated  this  decision  in  2002.  As  of  July 
1,  2003,  residents  in  all  specialties  are  required  to  adhere  to  an  80-hour  work  week 
(http://www.acgme.org/acWebsite/newsRoom/newsRm  dutyHours.asp').  This  mandate 
was  due,  in  large  part,  to  the  Institute  of  Medicine’s  (IOM)  report,  To  Err  is  Human: 
Building  a  Safer  Health  System,  which  focused  on  the  relationship  between  work  hours 
and  medical  error  (Kohn,  Corrigan,  &  Donaldson,  1999).  This  report  revealed  that 
medical  error  was  responsible  for  98,000  deaths  and  over  1  million  injuries  every  year  in 
the  United  States  alone  (Chen  et  al.,  2003).  Although  many  individuals  outside  the 
medical  field  have  criticized  the  long  hours  traditionally  required  by  surgical  programs  in 
the  past  (e.g.,  Botta,  2003),  there  are  some  surgeons  who  oppose  the  control  that  external 
bodies  have  recently  had  over  their  field.  There  is  a  growing  fear  that  the  ACGME 
mandate  will  result  in  unprepared  surgeons  and  thus,  an  overall  decline  in  patient  care 
(Komenaka,  2003).  Furthermore,  the  80-hour  restriction  impacts  continuity  in  care  since 
residents  may  not  be  able  to  see  the  patients  on  whom  they  operated  the  previous  day. 
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Although  it  is  not  yet  known  whether  the  80-hour  restriction  will  be  beneficial  or 
detrimental  to  surgeons  and  their  patients,  it  is  clear  that  this  mandate  demonstrates  the 
need  for  clearly  articulated  standards  of  surgical  performance.  The  Society  of  University 
Surgeons  (SUS)  released  a  statement  about  the  resident  work  hour  issue,  stating  that 
specific  hours  for  surgery  residents  cannot  be  arbitrarily  defined.  The  SUS  statement 
suggests  that  constrained  work  hours  “do  not  prepare  residents  for  the  real  world  of 
surgical  practice”  (Cole,  Bertagnolli,  &  Nussbaum,  2002).  Some  surgeons  also  fear  that 
the  mandate  will  affect  the  quality  of  future  surgeons.  For  instance,  Komenaka  (2003)  is 
concerned  that  a  very  different  type  of  medical  student  will  now  be  attracted  to  the 
surgical  professional.  He  argues  that  students  who  choose  “lifestyle  specialties”  (e.g., 
dermatology,  emergency  medicine,  anesthesia,  etc.)  may  now  be  attracted  to  surgical 
programs  because  time  commitments  and  training  obligations  are  less  burdensome  under 
the  new  mandate.  According  to  Komenaka,  individuals  who  are  attracted  to  lifestyle 
specialties  have  a  work  ethic  and  value  system  that  are  a  poor  match  for  surgery.  If  this 
is  indeed  the  case,  the  development  of  standards  of  competence  is  more  imperative  now 
than  before  the  introduction  of  the  80-hour  limit.  In  fact,  residency  programs  may  find  it 
necessary  to  devote  more  attention  to  the  selection  of  surgical  residents,  particularly  if 
greater  numbers  of  medical  students  decide  to  apply  to  surgery  programs. 

Selection  instruments  should  be  able  to  exclude  applicants  who  lack  not  only  the 
potential  to  master  the  important  technical  and  interpersonal  skills  needed  in  surgery,  but 
also  identify  those  applicants  lacking  the  personal  characteristics  and  values  that  many 
senior  surgeons  want  their  successors  to  possess.  To  rely  on  these  methods  for  important 


22 

selection  and  promotion  decisions,  it  is  essential  that  they  are  subjected  to  well- 
established  principles  of  validation. 

This  work  hour  mandate  places  considerable  demands  on  surgical  residency 
programs.  Program  directors  are  forced  to  work  within  the  constraints  of  an  80-hour 
week  and  still  produce  high  performing  surgeons.  In  addition,  educators  must  ensure 
their  residents  have  the  skills  required  to  perform  the  latest  surgical  techniques. 

New  Technologies  and  Skill  Sets 

As  with  the  introduction  of  any  new  technology,  the  adoption  of  MIS  required 
new  training  procedures.  By  the  late  1990s,  many  authors  had  discussed  the  need  for 
objective  assessment  and  feedback  parameters  in  MIS  (Sokollik,  Gross,  &  Buess,  2004). 
Although  MIS  was  a  revolutionary  advancement  within  the  surgical  community,  the 
training  it  requires  has  been  a  burden  for  most  surgical  programs.  Specifically,  MIS 
requires  many  new  skills  that  are  less  relevant  to  conventional  open  procedures  including 
keen  psychomotor  skills,  interpreting  the  three-dimensional  operating  field  from  a  two- 
dimensional  monitor,  and  adapting  to  reduced  tactile/haptic  feedback  (Hanna  et  al., 

1996).  Furthermore,  performing  advanced  MIS  procedures  typically  requires  surgeons  to 
undergo  an  additional  one  to  two  years  of  postresidency  training  (Gallagher,  Ritter,  & 
Satava,  2003).  Since  MIS  is  the  gold  standard  for  an  increasing  number  of  surgeries 
today  (e.g.,  cholecystectomy,  antireflux  surgery),  most  expert  practitioners  agree  that 
education  and  training  should  be  intensified  to  ensure  optimal  quality  of  treatment 
(Grantcharov,  Bardram,  Funch-Jensen,  &  Rosenberg,  2003). 

Surgical  educators  typically  acknowledge  that  the  complexity  associated  with 
MIS  must  be  handled  cautiously.  The  acquisition  of  these  new  skills  poses  a  serious 
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challenge  to  conventional  systems  of  surgical  training.  Accordingly,  the  establishment  of 
clear  standards  of  surgical  performance  in  MIS  will  not  be  easy.  However,  since  MIS  is 
associated  with  significantly  higher  complication  rates  (Deziel  et  al.,  1993),  particularly 
during  a  surgeon’s  early  use  of  these  procedures  (Moore  &  Bennett,  1995),  it  is 
imperative  that  education  and  training  in  laparoscopic  skills  play  an  important  role  in 
surgical  programs.  Similarly,  expert  surgeons  need  to  be  able  to  clearly  define  what 
makes  a  surgeon  skilled  in  MIS.  Once  surgeons  agree  upon  such  a  definition,  assessment 
and  feedback  should  become  a  regular  component  of  their  training  in  laparoscopic  skills. 

The  section  described  above  illustrates  some  of  the  difficulties  researchers  face  in 
trying  to  define  and  assess  a  surgeon’s  level  of  performance.  It  is  anticipated  that  the 
expert  participants  in  the  present  study  will  mention  these  themes  when  discussing  the 
qualities  that  impact  surgical  excellence. 

Competency  Modeling:  A  New  Approach  to  Determining  Levels  of  Surgical 

Performance 
What  are  Competencies? 

During  the  past  twenty  years,  economic,  demographic,  and  technological  changes 
have  had  a  powerful  influence  on  modem  organizations.  These  changes  have  posed  a 
major  challenge  for  organizations  to  remain  competitive  within  local  and  global  markets. 
Due  to  heavy  competition,  many  organizations  have  focused  on  developing  and 
maximizing  the  talent  of  their  human  resources.  Previously,  the  success  of  many 
organizations  was  defined  by  monetary  capital  and  hard  assets  (McLagan,  1997).  Today, 
employees’  knowledge,  skills,  abilities,  and  other  personal  characteristics  (KSAOs)  are 
considered  to  be  an  important  yardstick  for  measuring  an  organization’s  success. 
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In  order  to  become  a  high  performing  organization,  it  is  imperative  that  high 
performing  people  are  identified,  selected,  and  ultimately  retained  (Rodriguez,  Patel, 
Bright,  Gregory,  &  Gowing,  2002).  Over  the  past  three  decades,  many  human  resource 
specialists  have  begun  to  rely  on  competency  models  to  select  high  performers.  Although 
there  have  been  many  attempts  to  describe  competency,  one  widely  accepted  definition  is 
“an  underlying  characteristic  of  a  person  which  results  in  effective  and/or  superior 
performance  on  the  job”  (Klemp,  1980,  p.  21).  As  such,  competencies  are  also  referred 
to  as  “superior-performer  differentiators”  (McLagan,  1997)  or  “success  factors” 

(Mirabile,  1997).  Unlike  a  list  of  tasks  found  in  a  typical  job  description,  a  competency 
model  describes  the  talents  and  skills  a  person  must  have  to  be  successful  in  an 
organization  (Rodriguez,  Patel,  Bright,  Gregory,  &  Gowing,  2002). 

Competency  modeling  is  a  systematic  process  of  determining  the  combination  of 
KSAOs  needed  to  perform  at  a  high  level  in  an  organization  (Harris,  1998;  Lucia  & 
Lepsinger,  1999).  Therefore,  a  competency  model  is  a  general  description  of  the  qualities 
that  top-level  employees  possess.  Competency  models  may  differ  in  format  (e.g.,  verbal 
or  graphical)  as  a  function  of  the  data  collection  method  and  customer  requirements. 
Competency  models  can  be  used  for  selection,  training  and  development,  performance 
evaluation,  promotion  decisions,  leadership  development,  and  succession  planning  (Lucia 
&  Lepsinger,  1999).  Moreover,  competencies  help  to  ensure  that  employees  and 
managers  alike  share  a  common  view  of  what  it  takes  to  be  successful  in  an  organization. 

The  development  of  a  competency  model  is  a  complicated  procedure,  particularly 
if  a  job  involves  higher-level  cognitive  processes,  such  as  problem-solving  and  strategic 
thinking.  Those  KSAOs  that  are  concrete  are  much  easier  to  identify,  describe,  and 
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measure  than  those  that  are  abstract.  For  example,  although  it  may  be  straightforward  to 
develop  a  typing  test  that  measures  psychomotor  ability,  it  is  much  more  complicated  to 
measure  hypothetical  constructs  such  as  creativity.  Further,  characteristics  such  as 
aptitudes  and  personality  traits  are  typically  more  difficult  to  measure  than  knowledge, 
skills,  and  abilities  because  there  is  less  agreement  on  definitions  of  these  terms,  which  in 
mm,  complicates  efforts  at  validation.  Competency  modeling  can  be  helpful  for 
identifying  and  defining  these  abstract  personal  characteristics.  It  can  also  be  used  to 
classify  personal  characteristics  that  are  more  trainable  and  those  that  are  innate. 

There  are  several  reasons  why  competency  models  are  becoming  increasingly 
popular.  For  instance,  competency  models  improve  the  ability  of  selection  committees  to 
identify  applicants  who  possess  critical  KSAOs.  Competency  models  can  be  used  to 
improve  the  interview  process  by  ensuring  that  all  persons  involved  in  making  selection 
decisions  are  using  the  same  criteria.  In  addition,  competency  models  provide  an 
alternative  to  strong  reliance  on  “gut  instincts”  or  initial  impressions  gained  from 
employment  interviews,  which  have  historically  proven  to  be  a  weak  predictor  of  job 
success  (Latham  &  Wexley,  1981).  They  also  increase  the  likelihood  that  individuals 
selected  for  a  job  will  succeed.  Furthermore,  competency  models  can  minimize 
investment  in  employees  who  fail  to  reach  certain  levels  of  performance  (Lucia  & 
Lepsinger,  1999).  Because  it  can  be  extremely  costly  to  recruit,  hire,  and  train  an 
employee  such  as  a  surgical  resident,  competency  models  can  help  to  optimize  the 
process  of  selecting  the  best  candidates. 


26 

The  ACGME  Competencies 

Over  the  past  decade,  medical  schools  and  residency  programs  have  undergone 
many  changes.  For  instance,  the  Balanced  Budget  Act  of  1997  mandated  that  medical 
schools  revise  their  curricula  (Langdale,  Schaad,  Wipf,  Marshall,  &  Scott,  2003).  In 
1999,  the  same  year  that  the  IOM  released  its  report,  To  Err  is  Human  (Kohn,  Corrigan, 

&  Donaldson,  1999),  the  ACGME  took  radical  steps  to  define  competencies  in  resident 
education.  By  July  1,  2001,  the  ACGME  mandated  that  all  U.S.  residency  programs 
implement  a  curriculum  and  evaluation  program  based  on  the  six  core  competencies.  The 
ACGME  gave  residency  programs  a  ten-year  timeline  to  implement  the  new 
competencies  into  their  curriculum  and  use  them  to  guide  educational  goals,  objectives, 
and  evaluation  instruments  (http://www.aegme.org/outcome/).  The  American  Board  of 
Surgery  also  adopted  the  same  six  competencies  for  ongoing  assessment  of  surgeons 
(Dunnington  &  Williams,  2003).  The  six  ACGME  core  competency  requirements  are 
shown  in  Table  1  along  with  their  definitions  and  behavioral  examples. 
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Table  1 

ACGME  Competencies  and  Definitions _ 

Competency _ Definition _ 

Patient  Care  Patient  care  that  is  compassionate,  appropriate,  and 

effective  for  the  treatment  of  health  problems  and  the 
promotion  of  health. 

Examples: 

•  Able  to  develop  and  execute  patient  care  plans 
appropriate  for  the  resident’s  level 

•  Performs  procedures  safely  and  proficiently 

•  Demonstrates  proficiency  of  sterile  technique 

Medical  Knowledge  Medical  knowledge  about  established  and  evolved 

biomedical,  clinical,  and  cognate  (e.g.,  epidemiological 
and  social-behavioral)  sciences  and  the  application  of  this 
knowledge  to  patient  care. 

Examples: 

•  Critically  evaluates  and  demonstrates  knowledge  of 
pertinent  scientific  information 

•  Uses  clinical  data  in  operative  decision-making  and 
patient  management 

•  Demonstrates  appropriate  basic  and  clinical  science 
knowledge  (specifically  relating  to  anatomy, 
pathophysiology,  fluids  and  electrolytes,  and 
pharmacology) 

Practice-based  learning  and  improvement  that  involves 
investigation  and  evaluation  of  their  own  patient  care, 
appraisal,  and  assimilation  of  scientific  evidence,  and 
improvement  of  patient  care. 

Examples: 

•  Critiques  personal  practice  outcomes 

•  Integrates  scientific  evidence  when  diagnosing  and 
developing  treatment  plans 

•  Effectively  uses  information  technology  to  manage 
information 


Practice-Based  Learning 
and  Improvement 
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Table  1-  Continued 


Competency _ Definition _ 

Interpersonal  and  Interpersonal  and  communication  skills  that  result  in 

Communication  Skills  effective  information  exchange  and  collaboration  with 

patients,  their  families,  and  other  healthcare  professionals. 

Examples: 

•  Communicates  effectively  with  other  healthcare 
professionals 

•  Counsels  and  educates  patients  and  families 

•  Demonstrates  caring  behavior  toward  patients  and 
families 


Professionalism  Professionalism  as  manifested  through  a  commitment  to 

carrying  out  professional  responsibilities,  adherence  to 
ethical  principles,  and  sensitivity  to  a  diverse  population. 

Examples: 

•  Demonstrates  a  commitment  to  continuity  of  patient 
care 

•  Demonstrates  sensitivity  to  age,  gender,  and  culture  of 
patients  and  other  health  care  professionals 

•  Integrates  constructive  feedback 

System-Based  Learning  System-based  practice  as  manifested  by  actions  that 

demonstrate  an  awareness  of  and  response  to  the  larger 
context  and  system  of  health  care  and  effectively  call  on 
system  resources  to  provide  optimal  care. 

Examples: 

•  Practices  high  quality,  cost-effective  patient  care 

•  Demonstrates  an  understanding  of  the  role  of  different 
specialties  and  other  health  care  professionals  in  overall 
patient  management 

•  Able  to  access  and  mobilize  outside  resources 
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Assessing  Competencies  in  Medicine  and  Surgery 

The  term  “competency”  is  defined  differently  in  psychology  than  in  most  other 
fields.  In  medicine,  competency  is  typically  synonymous  with  the  terms,  “ability”  or 
“skill.”  Langdale  and  his  colleagues  (2003)  defined  competency  in  medicine  as  “having 
sufficient  experience  to  allow  the  independent  completion  of  the  task  with  minimal  or  no 
supervision”  (p.  40).  Further,  in  medicine,  competency  is  generally  measured  through 
volume  performance  and  patient  outcome,  while  surgery  may  also  use  performance 
measures  such  decision  time  and  reaction  time  (Trunkey  &  Botney,  2001). 

Psychologists,  in  contrast,  define  competencies  as  KSAOs  that  distinguish 
exceptional  from  average  performers  (Lucia  &  Lepsinger,  1999;  McClelland,  1973, 
Schippmann,  et  ah,  2000;  Spencer,  McClelland,  &  Spencer,  1994).  In  the  context  of 
surgery,  a  competency  based  on  the  psychological  definition  would  be  a  skill  or  attribute 
that  differentiates  an  exceptional  surgeon  from  a  typical  surgeon.  Although  the  medical 
and  psychological  definitions  of  competency  differ  slightly,  it  is  a  difference  worth 
noting.  In  medicine,  all  residents  are  expected  to  acquire  minimum  standards  of 
performance  in  each  of  the  six  ACGME  designated  competencies  throughout  their 
training.  Competencies,  as  described  in  1-0  psychology,  are  the  KSAOs  that  are  largely 
associated  with  outstanding  performers.  Thus,  they  describe  characteristics  that  are 
beyond  the  basic  minimum  standards  expected  of  residents  by  the  end  of  their  training. 

In  fact,  these  qualities  may  not  begin  to  manifest  until  a  surgeon  is  already  established  in 
his  or  her  career.  Therefore,  there  may  be  some  overlap  between  the  results  of  the 
present  study  and  the  ACGME  competencies,  but  it  is  likely  that  additional  KSAOs  will 
be  identified  that  are  unique  to  only  exceptional  surgeons. 
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Leadership 

Although  no  one  denies  that  the  ACGME  core  competencies  are  a  fundamental 
part  of  surgical  education  and  training,  several  surgical  educators  have  also  begun  to 
express  the  need  to  incorporate  basic  leadership  training  in  both  the  undergraduate  and 
graduate  educational  curricula  (Craven,  2002;  Itani,  Liscum,  &  Brunicardi,  2004; 
Schwartz,  1998).  According  to  Muller  (1984),  leadership  training  has  historically 
occurred  on  an  informal  basis  during  random  opportunities  or  via  observation  during 
surgical  training.  Today,  a  greater  number  of  surgical  programs  are  changing  their 
curricula  to  bolster  the  development  of  leadership  skills  to  a  more  prominent  level 
(Bowen,  1998;  Sims  &  Darcy,  1997). 

This  curricular  modification  is  imperative  because  many  surgical  residents  do  not 
feel  confident  or  competent  in  their  leadership  skills.  Itani  et  al.  (2004)  administered  a 
survey  to  43  general  surgery  residents  and  asked  them  to  rank  1 8  leadership 
characteristics  in  terms  of  importance  and  then  rate  their  own  levels  of  confidence  and 
competency  in  the  same  leadership  areas.  Although  nearly  all  (92%)  of  the  residents 
rated  the  leadership  characteristics  as  either  “somewhat”  or  “very  important”  to  their 
careers,  more  than  half  of  the  respondents  rated  themselves  as  either  “minimally”  or  “not 
competent”  in  10  of  the  18  leadership  areas.  Over  75%  of  the  residents  indicated  that 
they  were  least  competent  in  conflict  resolution,  practice  management,  and  leadership 
theory. 

The  type  of  leadership  required  in  surgery  may  be  different  from  that  of  many 
other  disciplines.  In  teaching  hospitals,  in  particular,  a  strict  formal  division  of  labor 
exists  with  the  attending  surgeon  at  the  top  of  the  hierarchy,  followed  by  a  chief  resident, 
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surgical  interns,  senior  medical  students,  and  finally,  junior  medical  students.  Since  the 
attending  surgeon  is  ultimately  responsible  for  everything  that  occurs  in  his  or  her 
surgical  service,  he  or  she  is  viewed  as  a  leader.  The  rank  of  an  attending  surgeon  grants 
almost  total  power  and  authority  to  address  issues  that  arise  within  his  or  her  service. 
Furthermore,  an  attending  also  holds  the  power  to  make  or  break  careers  (Bosk,  2003). 

Despite  the  strict  hierarchy  that  exists  in  surgery,  particularly  in  teaching 
hospitals,  actual  surgical  procedures  require  teamwork.  Because  the  practice  of  surgery 
is  team-oriented,  some  surgical  programs  are  introducing  collaborative  leadership 
training  (e.g.,  Awad,  Hayley,  Fagan,  Berger,  &  Brunicardi,  2004).  The  outcome  of  any 
surgical  procedure  is  contingent  on  the  entire  medical  team;  therefore,  it  is  imperative 
that  surgeons  not  only  know  how  to  collaborate  with  others,  but  also  how  to  lead  a 
surgical  team  (Gawande,  2001).  According  to  Awad  and  his  colleagues,  collaborative 
leadership  training  should  focus  on  teaching  effective  communication,  alignment  of  the 
team  toward  providing  good  patient  care,  and  integrity.  If  a  team  leader  has  integrity,  he 
or  she  will  hold  the  entire  team  to  high  personal  and  professional  standards.  Integrity  will 
also  help  to  create  an  atmosphere  of  trust  among  team  members. 

Some  fields  of  medicine,  including  cardiac  surgery  and  anesthesia,  have  adopted  a 
specific  type  of  team  training  known  as,  crew  resource  management  (CRM),  to  enhance 
both  communication  and  patient  safety  (Helmreich  &  Davies,  1997;  Howard,  Gaba,  Fish, 
Yang,  &  Samquist,  1992).  Crew  resource  management  was  originally  developed  in  the 
aviation  community  to  increase  safety  practices  and  reduce  errors  and  accidents  among 
cockpit  teams  (Salas,  Burke,  Bowers,  &  Wilson,  2001).  One  conclusion  of  the  IOM 
report  (1999),  To  Err  is  Human,  was  that  CRM  had  the  potential  to  improve  patient 


32 

safety  and  reduce  medical  errors.  Thus,  medical  educators  were  urged  to  translate  and 
apply  the  concepts  of  CRM  within  their  domain. 

Although  leadership  and  teamwork  skills  are  not  explicitly  listed  in  the  ACGME 
core  competencies,  an  increasing  number  of  surgical  educators  realize  that  they  want  to 
produce  surgeons  who  are  knowledgeable,  able  to  communicate  effectively,  and 
demonstrate  integrity  and  professionalism.  These  qualities  are  strongly  related  to  being 
an  excellent  leader  in  a  high  performing  team  (Larson  &  LaFasto,  1989). 

Intelligence 

Although  most,  if  not  all  of  the  ACGME  competencies  require  a  surgeon  to  be 
intelligent,  we  do  not  know  if  exceptional  surgeons  are  more  intelligent  than  the  typical 
surgeon.  In  other  words,  would  a  competency  model  of  an  exceptional  surgeon  deem 
intelligence  to  be  a  factor  that  distinguishes  different  levels  of  surgical  performance? 

The  study  of  intelligence  has  played  an  important  role  in  psychology  for  over  a 
century.  The  first  article  on  intelligence,  ‘“General  Intelligence,’  Objectively  Determined 
and  Measured”  was  written  by  Spearman  in  1904.  Since  the  appearance  of  this  article, 
several  major  theories  of  intelligence  have  emerged,  some  more  controversial  than  others. 
For  example,  Jensen  (1979)  maintained  the  existence  of  a  single,  general  factor  in  human 
intelligence.  This  factor  was  called  g,  or  general  mental  ability.  Jensen  believed  that  g 
was  identified  through  the  factor  analysis  of  correlations  among  several  different 
intelligence  tests.  Gardner  (1983),  on  the  other  hand,  argued  that  there  are  seven 
different  types  of  intelligences:  interpersonal,  intrapersonal,  linguistic,  logical- 
mathematical,  musical,  spatial,  and  bodily-kinesthetic.  In  an  effort  to  incorporate  several 
existing  theories  of  intelligence,  Sternberg  (1985)  developed  the  triarchic  theory  of 
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human  intelligence.  According  to  Sternberg  (1988),  intelligence  can  be  thought  of  as 
“the  purposive  adaptation  to,  selection  of,  and  shaping  of  real-world  environments 
relevant  to  one’s  life  and  abilities”  (1988,  p.  65).  Sternberg  argues  that  traditional 
conceptions  of  intelligence  are  too  narrow  and  that  an  expanded  view  of  the  construct  is 
necessary.  Accordingly,  Sternberg  developed  a  theory  that  divides  intelligence  into  three 
different,  yet  interrelated  parts. 

Analytical  Intelligence 

Analytical  intelligence  was  the  first  module  described  by  Sternberg  and  arguably 
the  most  thoroughly  developed  (Tigner  &  Tigner,  2000).  It  is  often  measured  through 
academic  problem  solving,  analogies,  and  reading  comprehension  exercises.  Sternberg 
(1985)  believes  that  analytical  intelligence  is  based  on  a  combination  of  metacomponents, 
performance  components,  and  knowledge  acquisition  components  of  intelligence. 
Metacomponents  include  planning,  monitoring,  and  evaluating  skills  that  aid  in  problem 
solving.  Sternberg  considers  metacomponents  to  be  the  higher-order  executive  processes 
of  human  intelligence.  Metacomponents  are  viewed  as  the  foundation  of  mental  self¬ 
management.  They  allow  an  individual  to  decide  what  cognitive  action  to  take  and  which 
components  are  necessary  to  perform.  Performance  components  are  the  basic  operations 
required  to  execute  any  cognitive  action.  It  would  be  nearly  impossible  to  list  and 
describe  all  performance  components  (Sternberg,  1988).  However,  some  examples 
include  encoding  and  maintaining  information  in  short-term  memory,  discovering 
relationships  between  objects  or  events,  performing  mental  calculations,  applying 
knowledge  to  real  world  problems,  and  critiquing  ideas  and  products.  Finally,  knowledge 
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acquisition  components  are  processes  used  to  gain  and  store  new  knowledge  (e.g.,  the 
way  in  which  individuals  learn  new  words  and  concepts). 

An  individual  with  high  analytical  intelligence  typically  performs  well  in 
traditional  educational  settings  (Tigner  &  Tigner,  2000).  Since  standard  intelligence  tests 
(e.g.,  the  Scholastic  Aptitude  Test  or  SAT)  were  specifically  developed  to  predict 
academic  performance,  it  is  logical  that  analytical  intelligence  is  most  similar  to  the  type 
of  intelligence  measured  in  academic  settings.  Individuals  with  high  analytical 
intelligence  are  likely  to  be  good  academic  test-takers  and  thus,  viewed  as  “smart.”  They 
also  tend  to  be  good  critical  thinkers. 

Creative  Intelligence 

Sternberg  (e.g.,  1985,  1988)  also  noted  that  although  many  individuals  appear  to 
be  gifted  in  traditional  analytical  and  memory  skills,  they  are  not  necessarily  adept  at 
creating  their  own  ideas.  Likewise,  there  are  many  individuals  who  excel  at  developing 
new  and  imaginative  ideas,  but  do  not  score  well  on  standard  intelligence  tests.  It  would 
therefore  seem  that  creativity  extends  beyond  traditional  intelligence.  Thus,  the  second 
facet  of  intelligence  is  creative  intelligence  or  the  experiential  facet  of  intelligence. 
According  to  Sternberg  (1985),  creativity  involves  insight,  synthesis,  and  the  ability  to 
react  to  novel  situations  and  stimuli.  Creative  behavior  is  demonstrated  when  a  person 
must  deal  with  a  relatively  novel  task  or  situation,  or  when  a  task  becomes  automatized. 
The  triarchic  theory  dictates  that  performance  on  novel  tasks  is  an  excellent  measure  of 
intelligence  because  it  assesses  an  individual's  ability  to  apply  existing  knowledge  to  new 
problems.  For  instance,  when  individuals  first  arrive  in  a  foreign  country,  they  must 
rely  on  their  intelligence  to  adjust  to  the  demands  of  a  new  culture.  However,  the  less 


35 


attention  required  to  process  novel  conditions,  the  more  that  remains  to  automatize 
actions.  Likewise,  the  more  efficiently  an  individual  can  automatize  performance,  the 
more  attention  he  or  she  will  have  for  dealing  with  novel  situations  and  tasks  (Sternberg, 
1988).  Sternberg  provides  an  excellent  example  of  the  relationship  between  coping  with 
novelty  and  automatization  of  information  processing.  Returning  to  the  example  of 
traveling  to  a  foreign  country,  the  less  attention  an  individual  devotes  to  adjusting  to  the 
new  currency,  the  more  attention  he  or  she  will  have  to  enjoy  shopping. 

Individuals  high  in  creative  intelligence  excel  at  discovering,  creating,  and 
inventing  ideas  and  products  (Sternberg,  Castejon,  Prieto,  Hautamaki,  &  Grigorenko, 
2001).  One  must  note  that  creativity  is  context-specific;  thus,  what  is  creative  in  one 
context  may  not  be  creative  in  another  (Sternberg  &  Lubart,  1995).  Sternberg  believes 
that  it  is  essential  to  view  creativity  as  a  property  of  an  individual  as  he  or  she  interacts 
with  one  or  more  systems  (Sternberg,  2001).  For  instance,  in  the  1950s,  certain  New 
York  abstract  expressionist  artists  such  as  Jackson  Pollock  or  Willem  de  Kooning  were 
the  first  to  produce  drip  style  paintings.  At  the  time,  this  style  was  considered  to  be 
innovative  because  it  represented  a  major  change  in  the  art  world.  By  contrast,  if 
someone  were  to  produce  a  drip  painting  today,  it  would  not  be  considered  creative 
because  this  style  is  no  longer  original.  In  essence,  Sternberg  believes  that  highly 
creative  behavior  is  “crowd  defying.”  In  other  words,  creative  individuals  are  often 
underappreciated  or  even  attacked  for  their  way  of  thinking.  If  the  public  eventually 
decides  to  accept  these  novel  ideas,  crowd-defying  creativity  actually  may  propel  a  field 
in  an  entirely  new  direction  (Sternberg  &  Lubart,  1995). 


36 


Practical  Intelligence 

Sternberg  (1985)  was  not  the  first  author  to  discuss  practical  intelligence.  Neisser 
(1976)  distinguished  academic  intelligence  from  intelligent  performance  in  natural 
settings.  The  former  term  is  comparable  to  Sternberg’s  analytical  intelligence  while  the 
latter  is  equivalent  to  his  concept  of  practical  intelligence.  For  both  Sternberg  and 
Neisser,  traditional  intelligence  is  insufficient  for  successful  performance  in  real-world 
situations  because  it  does  not  address  the  ability  to  learn  information  and  then  apply  it 
correctly.  Practical  intelligence  or  the  contextual  aspect  of  intelligence,  also  refers  to  the 
ability  to  grasp,  understand,  and  deal  with  everyday  tasks,  particularly  in  regard  to  one’s 
short-term  and  long-term  goals  (Sternberg,  1988).  More  commonly,  practical 
intelligence  is  conceived  of  as  “street  smarts”  or  intelligence  that  functions  in  the  real 
world.  Traditional,  academic  intelligence  is  only  a  subset  of  the  abilities  needed  to 
determine  how  a  person  performs  in  real-life  contexts. 

Unlike  the  other  two  aspects  of  intelligence,  practical  intelligence  includes 
attitudes  and  emotions  that  may  be  associated  with  real-world  intelligence.  For  example, 
although  the  decision  to  purchase  a  house  is  partly  an  intellectual  decision,  this  decision 
also  has  a  strong  emotional  component  for  most  people.  Another  novel  aspect  of 
practical  intelligence  is  that  it  is  very  individualistic,  particularly  in  comparison  to 
analytical  intelligence.  Practical  intelligence  allows  people  to  take  the  knowledge  they 
have  gained  throughout  their  many  life  experiences  and  generalize  that  knowledge  to 
other  contexts  they  encounter.  It  is  also  distinctive  in  that  what  is  relevant  and  useful  for 
one  person’s  life  may  be  totally  irrelevant  for  another  individual  (Tigner  &  Tigner,  2000). 
Consequently,  practical  intelligence  is  difficult  to  assess  through  traditional  intelligence 
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tests  and  shows  little  or  no  correlation  with  analytical  or  creative  intelligence  (Sternberg, 
1988). 

A  Competency  Model  for  Exceptional  Surgeons 

The  ACGME  core  competencies  have  served  to  either  enhance  or  develop  new 
curricula  for  surgical  programs  (Dunnington  &  Williams,  2003).  Generally,  these 
competencies  have  not  been  used  to  create  new  selection  systems  or  evaluation  plans.  In 
fact,  there  is  no  standardized  approach  to  selecting  prospective  surgeons.  Rather, 
residency  directors  use  a  variety  of  criteria  in  their  selection  decisions.  According  to 
Bemer,  Brooks,  and  Erdmann  (1993),  the  majority  of  surgical  residency  programs  rely  on 
face-to-face,  unstructured  interviews,  academic  transcripts,  and  scores  on  the  National 
Board  of  Medical  Examiners  (NBME)  Part  I  and  Part  II  examinations.  Although  most 
selection  committees  are  currently  interested  in  academic  indicators  and  interview 
performance,  it  is  important  to  note  that  interviews  have  a  long  and  tainted  history  from 
the  1-0  psychology  perspective.  Interviews  suffer  from  low  reliability  (Judge,  Higgins, 

&  Cable,  2000).  Interviews  are  also  time  and  labor  intensive  (Cascio,  1998),  prone  to 
interview  bias  (e.g.,  Cash,  Gillen,  &  Bums,  1977;  Beehr  &  Gilmore,  1982),  and  require 
extensive  training  with  active  practice  and  feedback  (Latham  &  Wexley,  1981). 
Individuals  who  conduct  interviews  should  not  only  be  aware  of  the  potential  problems 
inherent  in  the  interview  process,  but  also  the  best  methods  for  enhancing  validity  and 
legal  defensibility. 

In  many  organizations  outside  of  medicine,  standardized  psychological  tests  have 
been  used  for  decades  to  improve  their  selection  systems.  In  surgery  programs  within  the 
U.S.,  the  use  of  psychometrics  to  enhance  the  selection  process  is  just  beginning  to  gain 
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popularity.  International  surgical  programs,  however,  appear  to  be  slightly  ahead  of  the 
U.S.  in  using  this  approach.  For  instance,  in  1983,  the  Association  of  Surgeons  in  The 
Netherlands  integrated  psychometric  testing  in  their  training  process.  The  Royal  College 
of  Surgeons  is  also  considering  a  similar  approach  (Gilligan,  Treasure,  &  Watts,  1996). 
However,  the  main  problem  of  identifying  the  competencies  of  an  excellent  surgeon 
remains  and  until  this  problem  is  resolved,  the  selection  of  outstanding  surgeons  will  be 
greatly  hindered  (Wanzel,  Ward,  &  Reznick,  2002). 

A  competency  model  for  surgeons  could  be  used  by  selection  committees  to  more 
effectively  screen  for  surgical  acumen.  Further,  the  competency  model  would  also  be 
useful  in  identifying  those  candidates  who  are  ill-suited  for  surgical  residency.  In 
addition,  a  competency  model  can  be  used  to  improve  training.  In  other  words,  the  same 
set  of  competencies  that  are  used  for  selecting  high-quality  candidates  can  also  be  used  to 
guide  surgical  training  and  education.  The  use  of  a  competency  model  as  the  foundation 
for  a  training  program  can  ensure  that  training  is  based  on  the  main  KSAOs  that 
constitute  exceptional  performance  rather  than  passing  fads  that  training  practitioners 
choose  to  adopt  (Lucia  &  Lepsinger,  1999).  Additionally,  competency  models  can  serve 
to  make  training  comprehensive,  standardized,  and  directly  linked  to  performance 
evaluations. 

Finally,  the  use  of  a  competency  model  in  surgery  can  help  to  develop  leadership 
programs.  Few  surgical  residents  are  equipped  with  the  skills  and  the  know-how  to 
become  strong  leaders.  A  competency  model  can  be  used  to  develop  the  content  for 
leadership  training,  which  would  set  expectations  for  residents  in  their  roles  as  leaders 
and  help  them  identify  their  own  developmental  goals  (Lucia  &  Lepsinger,  1999). 
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Objectives  of  the  Present  Study 

The  primary  goal  of  the  present  study  was  to  develop  an  understanding  of  the 
nature  of  surgical  excellence  by  means  of  the  competency  modeling  approach.  In  the 
first  part  of  the  procedure,  outstanding  surgeons  representing  various  specialties  were 
interviewed  to  determine  the  specific  competencies  needed  to  be  considered  exceptional. 
In  addition  to  outstanding  surgeons,  residents  from  various  surgical  specialties  were 
interviewed.  When  developing  a  competency  model,  it  is  common  practice  to  compare 
the  interview  results  of  exceptional  performers  to  those  of  more  typical  performers. 

Since  it  would  be  difficult  to  identify  surgeons  who  would  identify  themselves  as 
“average,”  an  alternative  approach  was  used  targeting  less  experienced  surgeons. 
Furthermore,  most  competency  assessment  practitioners  interview  a  greater  number  of 
exceptional  performers  than  average  performers.  McClelland  (1998),  one  of  the  earliest 
developers  of  competency  modeling,  recommends  interviewing  exceptional  and  average 
performers  in  a  ratio  of  3:2.  He  believes  that  there  are  more  ways  to  be  exceptional  than 
there  are  to  be  typical.  This  ratio  was  also  used  in  the  present  study. 

The  purpose  of  collecting  data  from  the  surgical  residents  was  to  determine  how 
less  experienced  surgeons  describe  successful  job  performance.  Therefore,  another  goal 
of  the  present  study  was  to  identify  patterns  within  the  data  that  distinguish  between 
exceptional  and  novice  surgeons.  It  was  anticipated  that  although  some  degree  of  overlap 
would  be  found  among  the  competencies  provided  by  each  group  of  surgeons,  on  the 
whole  they  would  look  quite  different.  Since  the  exceptional  group  of  surgeons  would 
have  many  more  professional  experiences  to  draw  upon,  they  were  expected  to  have  a 
better  grasp  of  those  KSAOs  needed  to  become  truly  exceptional  in  any  field  of  surgery. 
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Predictions  of  the  Present  Study 

ACGME  Competencies 

It  was  also  anticipated  that  the  exceptional  surgeons  would  mention  the 
importance  of  ACGME  core  competencies  in  surgical  training;  however,  not  all  six  were 
expected  to  be  included  in  the  final  competency  model.  Since  all  surgical  residents  are 
expected  to  meet  the  minimum  standards  of  performance  described  by  the  ACGME  core 
competencies,  it  is  unlikely  they  would  differentiate  between  typical  surgeons  and  those 
who  are  truly  exceptional. 

Leadership  Skills 

Although  leadership  skills  are  not  explicitly  listed  in  the  ACGME  core 
competencies,  an  increasing  number  of  surgical  educators  recognize  the  importance  of 
producing  surgeons  who  are  knowledgeable,  able  to  communicate  effectively,  and 
demonstrate  integrity  and  professionalism.  As  stated  earlier,  an  increasing  number  of 
surgical  programs  are  changing  their  curricula  to  address  leadership  skills  (Bowen,  1998; 
Sims  &  Darcy,  1997).  Therefore,  it  was  anticipated  that  leadership  skills  would  be  a 
factor  that  distinguishes  an  exceptional  surgeon  from  an  average  one. 

Intelligence 

Intuitively,  one  might  surmise  that  an  exceptional  surgeon  would  also  be  highly 
intelligent.  Although  it  is  possible  that  the  experts  would  view  intelligence  as  an 
important  factor  for  average  and  exceptional  surgeons  alike,  it  was  anticipated  that  their 
descriptions  of  an  outstanding  surgeon  would  include  all  three  forms  of  intelligence  as 
described  by  Sternberg  (1985). 
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Critical  Thinking  Skills  and  Decision  Making  Ability 

Given  that  an  important  part  of  a  surgeon’s  job  is  to  make  complicated,  real-time 
decisions,  exceptional  surgeons  must  be  able  to  rapidly  evaluate  multiple  course  of 
treatment.  They  must  be  able  to  quickly  integrate  information  regarding  medication, 
diseases,  and  changes  in  a  patient’s  status.  Therefore,  it  was  anticipated  that  critical 
thinking  and  decision  making  ability  would  be  included  in  the  competency  model  of  an 
exceptional  surgeon. 

Technical  Skill 

Historically,  technical  skill  has  been  closely  linked  to  surgical  excellence.  With 
the  advent  of  MIS  procedures,  technical  skill  may  become  a  more  critical  component  of 
success  in  the  field.  Accordingly,  it  was  predicted  that  the  experts  would  include 
technical  skill  in  the  competency  model. 

Interpersonal  Skills 

Finally,  patients’  opinions  have  become  a  concern  for  the  medical  professional. 
As  stated  earlier,  patient  satisfaction  questionnaires  are  becoming  more  common  in  the 
assessment  of  medical  practitioners.  It  was  therefore  anticipated  that  the  experts  in  the 
present  study  would  indicate  that  surgeons’  attitudes  and  sensitivity  toward  their  patients, 
as  well  as  the  quality  of  their  interpersonal  skills,  would  be  directly  related  to  surgical 


excellence. 


42 


METHOD 

Participants 

The  participants  were  forty  surgeons  from  different  geographical  regions  in  the 
United  States.  Participants  were  divided  into  two  separate  groups.  The  first  group 
consisted  of  exceptional  surgeons.  These  surgeons  were  well-known  and  well-respected 
in  their  fields.  All  were  board-certified  and  fellows  of  the  American  College  of  Surgeons 
(ACS).  Since  it  is  difficult  to  determine  a  list  of  objective  criteria  for  surgical  excellence, 
subject  matter  experts  identified  surgeons  to  be  recruited  for  the  present  study.  An 
experienced  trauma  surgeon  and  three  of  his  colleagues  developed  a  list  that  included  a 
cross  section  of  surgeons  representing  different  surgical  specialties.  The  second  group  of 
participants  consisted  of  surgical  residents  from  various  specialties.  Since  these  surgeons 
were  in  the  initial  stages  of  their  career,  they  were  considered  novices.  Due  to  their  lack 
of  experience,  the  second  group  was  neither  board-certified  nor  fellows  of  the  ACS. 

Expert  participants  included  one  female  and  25  males  with  13  to  44  years  (M= 
29.0)  of  surgical  experience.  The  expert  sample  represented  seven  surgical 
subspecialties:  cardiac/thoracic  (N=  5),  critical  care  (N=  3),  gastro-intestinal  ( N  =  2), 
general  surgery  (. N  =  8),  head  and  neck  (N=  1),  surgical  oncology  (N=  5),  and  urology 
(N=  2).  Novice  participants  included  seven  males  and  seven  females  with  one  to  five 
years  of  surgical  experience:  PGY-1  (N=  2),  PGY-2  (N=  2),  PGY-3  (N=  2),  PGY-4  (N 
=  4),  PGY-5  (N=  4).  The  novice  sample  included  five  surgical  subspecialties:  general 
surgery  (N=  10),  orthopedic  (N=  1),  otolaryngology  ( N  =  1),  gynecological  oncology  ( N 
=  1),  and  urology  ( N  =  1). 


43 


Procedure 

Once  participants  in  both  groups  were  identified,  they  were  sent  a  formal  letter  via 
e-mail  that  explained  the  study  and  requested  their  participation  (See  Appendix  A). 
Participants  who  agreed  to  be  interviewed  were  sent  the  questions  several  days  before  the 
interview  was  scheduled  so  they  could  start  thinking  about  their  responses.  A  structured, 
one-on-one  interview  was  used  with  all  participants.  Interviews  were  approximately 
forty  minutes  in  length,  or  longer  if  the  surgeon’s  schedule  permitted.  All  participants 
granted  permission  to  have  the  interview  session  tape  recorded.  Since  the  terminology 
surrounding  a  surgeon’s  job  is  highly  technical,  the  interviews  were  tape  recorded  to 
ensure  the  accuracy  and  completeness  of  their  comments  (Lucia  &  Lepsinger,  1999). 
Detailed  hand-written  notes  were  taken  throughout  each  interview  as  well. 

All  participants  were  told  that  their  responses  would  be  kept  confidential  and 
anonymous.  They  were  instructed  to  refrain  from  mentioning  other  surgeons  or 
healthcare  professionals  by  name  during  the  interview.  Participants  were  assured  that 
their  names  would  not  be  linked  to  any  comments  included  in  the  written  report  from  the 
present  study.  The  present  study  received  approval  from  the  Institutional  Review  Boards 
of  Eastern  Virginia  Medical  School  and  Old  Dominion  University. 

Instruments 

A  structured  interview  was  developed  for  the  current  study  in  consultation  with  I- 
O  psychologists  who  have  extensive  experience  developing  competency  models,  as  well 
as  existing  literature.  The  interview  began  with  a  brief  introductory  session  explaining 
the  purpose  of  the  project  and  its  potential  impact.  Participants  were  told  why  they  were 
selected  to  participate  and  the  expected  duration  of  the  interview.  They  were  then  given 
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an  opportunity  to  ask  questions.  All  individuals  were  thanked  for  their  participation  both 
at  the  start  and  conclusion  of  the  interview.  Participants  in  both  groups  received  the  same 
structured  interview.  The  actual  interview  consisted  of  seven  main  questions.  The  list  of 
questions  in  the  order  presented  were  as  follows: 

1 .  Think  of  some  of  the  finest  surgeons  you  have  ever  known.  What  personality 
traits  do  you  believe  contributed  to  this  person’s  success? 

2.  Once  again,  think  of  a  surgeon  that  you  respect  and  admire.  What  values  or 
attitudes  contributed  to  this  person’s  success  as  a  surgeon? 

3.  Think  of  the  finest  surgeon  you  have  ever  known.  Now  describe  in  detail  one 
incident  that  reflects  the  personality,  values,  or  skills  that  contribute  to  their 
excellence. 

4.  Think  of  a  surgeon  with  whom  you  would  never  want  to  work.  Now  describe  in 
detail  one  incident  that  reflects  the  personality,  values,  or  lack  of  skills  that 
contributed  to  their  poor  performance  as  a  surgeon. 

5.  Think  of  some  of  the  finest  surgeons  that  you  know.  What  characteristics  or 
attributes  do  you  share  with  these  surgeons?  What  characteristics  or  attributes,  if 
any,  set  you  apart  from  these  exceptional  surgeons? 

6.  Describe  a  critical  or  significant  experience  that  represents  what  you  do  well  as  a 
surgeon  (i.e.,  an  experience  that  made  you  proud).  What  about  this  experience 
makes  you  feel  particularly  effective? 

7.  Finally,  please  describe  the  job  of  a  surgeon  in  the  future  (e.g.,  in  the  year  2015). 
What  knowledge,  skills,  or  other  attributes  will  a  surgeon  in  the  future  need  to  be 
exceptional? 
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For  each  question,  participants  were  probed  for  additional  information.  Depending 
on  the  situation,  probes  included: 

•  What  was  the  first  thing  you  (or  the  surgeon  in  question)  did? 

•  What  were  you  (or  the  surgeon  in  question)  trying  to  accomplish? 

•  What  were  you  (or  the  surgeon  in  question)  thinking  at  that  point? 

•  Tell  me  more  about  that. 

•  Can  you  think  of  a  specific  example  of  that? 

•  Is  there  anything  else  you  would  like  to  add  about  what  you  (or  the  surgeon  in 
question)  did  in  that  situation? 

All  interviews  were  transcribed.  The  transcriptions  allowed  for  careful  analysis  of 
interview  content  for  patterns  within  the  data. 

Analyses 

The  data  for  the  exceptional  surgeons  and  the  surgical  residents  were  handled 
differently.  The  analysis  of  the  exceptional  surgeons’  data  is  described  first.  A  common 
approach  for  analyzing  the  comments  derived  from  competency  interviews  is  known  as 
the  critical  incident  technique  (CIT;  Flanagan,  1954).  Although  this  technique  was 
developed  half  a  century  ago,  it  continues  to  be  used  in  both  practical  and  research 
settings  today  (Aamodt,  Keller,  Crawford,  &  Kimbrough,  1981;  Stitt-Gohdes,  Lambrecht, 
&  Redmann,  2000).  Bailey  and  Merritt  (1995)  argue  that  this  method  is  optimal  for 
rendering  holistic  and  professionally-oriented  descriptions  of  a  job.  Analyses  were 
performed  using  an  approach  based  on  the  critical  incident  technique  (CIT).  On  the  basis 
of  a  thorough  literature  review,  previous  medical  research,  and  the  analysis  of  the 
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transcripts,  the  responses  to  each  question  were  reduced  to  behavioral  or  critical 
incidents.  Flanagan  (1954)  referred  to  this  phase  as  incident  segmentation. 

In  the  second  phase,  the  critical  incidents  obtained  from  the  exceptional  surgeons 
were  sorted  into  categories  (i.e.,  competencies).  For  each  competency,  definitions  based 
on  the  critical  events  were  developed.  After  the  definitions  were  considered  complete, 
they  were  submitted  to  a  group  of  five  experienced  surgeons  to  be  confirmed  or  modified. 
These  surgeons  were  also  well-known  and  well-respected  in  their  respective  specialties. 
This  group  of  subject  matter  experts  was  responsible  only  for  revising  the  competency 
definitions. 

Next,  a  second  1-0  psychologist  with  experience  in  competency  modeling  and 
qualitative  analysis  sorted  the  critical  incidents  into  their  appropriate  categories.  His 
results  were  compared  to  the  categories  formed  in  Phase  two.  In  the  final  phase,  the 
importance  of  each  category  was  estimated.  Using  the  process  described  by  Aamodt  and 
his  colleagues  (1981),  the  number  of  critical  incidents  in  each  category  retained  after  the 
final  phase  was  summed.  The  number  of  critical  incidents  in  each  category  was  provided 
in  an  effort  to  establish  relative  importance  of  each  competency. 

In  addition  to  describing  the  characteristics  of  outstanding  surgeons,  the  interview 
data  were  also  used  to  describe  the  characteristics  of  poor  surgeons,  as  well  as 
requirements  for  exceptional  surgeons  in  the  future.  The  same  process  of  analysis  used 
for  the  expert  comments  was  applied  to  these  categories  of  comments. 

Interview  Data  from  the  Surgical  Resident  Group 

A  second  goal  of  the  present  study  was  to  identify  patterns  within  the  data  that 
distinguish  exceptional  from  novice  surgeons.  Once  again,  the  critical  incidents  obtained 
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from  the  residents  were  independently  sorted  into  categories  (i.e.,  competencies). 
Additionally,  relative  importance  ratings  were  determined  by  summing  the  critical 
incidents  within  each  competency.  Once  the  categories  were  considered  complete,  they 
were  not  presented  to  another  group  of  experienced  surgeons  for  further  consideration. 
The  objective  of  analyzing  the  surgical  resident  data  was  to  compare  and  contrast  the 
resultant  model  with  the  one  derived  from  the  exceptional  surgeons.  Therefore,  the 
interview  content  from  the  residents  did  not  impact  the  final  competency  model  of  an 
exceptional  surgeon.  Rather,  similarities  and  discrepancies  between  both  groups  of 
surgeons  are  described  and  explored  below. 
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RESULTS 
Main  Analysis 

Two  sets  of  interviews  were  performed:  the  main  analysis  centered  on  the  expert 
comments  and  the  secondary  analysis  addressed  the  resident  comments.  The  results  of 
the  main  analysis  will  be  described  first.  The  expert  comment  analysis  is  divided  into 
three  sections.  The  first  section  consists  of  the  characteristics  of  an  exceptional  surgeon 
(i.e.,  the  competencies).  The  second  section  consists  of  the  characteristics  of  a  poor 
surgeon  and  the  final  section  consists  of  the  characteristics  of  the  exceptional  surgeon  of 
the  future. 

Characteristics  of  an  Exceptional  Surgeon 

The  comment  analysis  of  the  characteristics  of  an  exceptional  surgeon  consisted 
of  298  individual  comments  made  by  26  expert  surgeons.  Each  comment  was  placed  into 
17  themes.  However,  an  a  priori  decision  was  made  to  include  only  themes  that 
contained  comments  from  a  minimum  of  eight  participants.  Any  theme  containing 
comments  from  fewer  than  eight  participants  was  not  considered  to  be  characteristic  of 
the  participant  pool  since  it  represented  less  than  one-third  of  the  total  number  of 
participants  (See  Appendix  B  for  the  residual  comments).  This  criterion  yielded  ten 
competencies  based  on  237  comments.  The  most  frequently  noted  competency  had  39 
comments  and  the  least  frequently  noted  competency  had  1 5  comments. 

Inter-rater  Agreement 

Two  1-0  psychologists  with  experience  in  competency  modeling  and  qualitative 
analysis  independently  sorted  the  237  comments  into  the  ten  competencies.  Cohen’s 
kappa  (Cohen,  1960)  was  used  to  provide  a  measure  of  inter-rater  agreement  between  the 
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two  judges.  The  value  of  kappa  for  this  data  set  was  0.85,  with  a  standard  error  of  .03. 
Many  researchers,  including  Landis  and  Koch  (1977),  believe  that  a  kappa  value  of  .61- 
.80  indicates  “substantial  agreement”  and  a  value  of  .81-1.00  indicates  “almost  perfect 
agreement.”  Therefore,  in  the  present  study,  it  can  be  said  with  confidence  that  the  two 
raters  had  extremely  high  agreement  on  their  sortings. 

The  Ten  Competencies 

The  following  section  contains  a  description  of  the  ten  competencies,  along  with 
the  total  number  of  individual  comments  comprising  each  competency.  Definitions  were 
created  for  each  competency  and  these  are  shown  in  Table  2.  Immediately  following  the 
table,  a  more  thorough  description  of  each  competency  is  provided. 


Table  2 

Ten  Competencies  of  an  Exceptional  Surgeon 

Competency _ #  of  Comments  Definition _ 

Dedication  to  Patient  Care  39  Consistently  and  passionately  regarding  the 

patient  as  the  most  important  part  of  any 
professional  transaction.  Maintaining  the 
patient  as  the  primary  focus  while 
simultaneously  holding  one's  personal 
needs,  ego,  and  obligations  as  secondary. 
Answering  all  of  a  patient's  questions  and 
concerns  while  always  communicating  that 
everything  possible  will  be  done  to  ensure  a 
positive  outcome. 

Integrity  33  Honestly  recognizing  one's  strengths  and 

weaknesses  as  a  surgeon.  Admitting  when 
one  is  wrong  and  publicly  accepting 
responsibility  for  mistakes.  Agreeing  to 
perform  operations  only  after  the  complete 
consideration  of  a  patient's  full  medical 
profile. 
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Table  2-  Continued 


Competency _ #  of  Comments  Definition _ 

Tireless  Work  Ethic  29  Displaying  a  high  level  of  energy  and  drive 

when  dealing  with  patients,  residents,  and 
other  healthcare  professionals.  Willing  and 
eager  to  work  long  hours  to  ensure 
excellence  in  one’s  career. 


Preparedness 


27 


Intellectual  Giftedness  23 

and  Curiosity 


Humility  21 


Compassion  19 


Devotion  to  Field  16 


Delivering  patient  care  in  a  thoughtful, 
precise,  and  deliberate  manner.  Consistently 
anticipating  complications  before  entering 
the  OR  and  having  multiple  contingencies 
covered.  Conducting  every  operation  in  a 
structured  and  orderly  way  as  to  not  confuse 
team  members. 

Displaying  an  ability  to  process  an 
enormous  amount  of  medical  information 
and  grasp  new  concepts  very  quickly. 
Maintaining  an  insatiable  curiosity  about  the 
field  through  the  conduct  of  research  and  the 
development  of  new  and  improved  surgical 
methods  and  procedures.  Tackling  medical 
issues  in  an  innovative  and  creative  manner. 

Displaying  respect  for  others  regardless  of 
their  rank  within  the  hospital  setting. 
Maintaining  a  modest  demeanor  when 
interacting  with  patients  and  other 
healthcare  professionals. 

Adopting  a  humanistic  approach  to  treating 
patients,  responding  in  a  caring  and 
empathetic  manner  to  their  suffering. 
Maintaining  a  gentle  and  patient  attitude 
with  co-workers  and  other  healthcare 
professional,  including  nurses,  residents  and 
medical  students. 

Demonstrating  an  intense  devotion  to  the 
field  of  surgery  disproportionate  to  other 
surgeons.  Viewing  surgery  not  as  a  job  but 
as  a  calling. 
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Table  2-  Continued 


Competency 

#  of  Comments 

Definition 

Rapid  Decision  Making 

15 

Demonstrating  the  capacity  to  quickly 
analyze  the  risks  and  benefits  of  multiple 
courses  of  treatment  and  to  choose  the  most 
appropriate  and  expeditious  path  to  a 
positive  patient  outcome.  Making  swift  yet 
informed  decisions  under  intense  pressure 
and  with  less  than  optimum  data  available. 

Passion  for  Teaching 

15 

Maintaining  the  utmost  dedication  to  the 
education  of  patients,  the  community  or 
other  health  professionals,  as  well  residents, 
both  in  and  out  of  the  classroom. 

Disregarding  any  time  limit  when  training 
others.  Demonstrating  an  intense 
responsibility  and  passion  for  creating  the 
next  generation  of  highly  qualified  surgeons. 

Competency  #1 :  Dedication  to  Patient  Care 

Twenty-two  expert  participants  defined  the  exceptional  surgeon  as  one  who  is 
totally  dedicated  to  patient  care.  In  fact,  39  comments  related  to  the  dedication  to  patient 
care  were  provided  by  the  participants.  An  essential  aspect  of  this  competency  is  the 
ability  and  willingness  to  treat  the  patient  with  the  utmost  importance  and  urgency. 
Consequently,  the  needs  of  a  surgeon,  including  any  personal  obligations,  are  viewed  as 
secondary  to  those  of  the  patient.  One  particularly  relevant  example  given  by  a 
participant  was  that  of  an  accomplished  surgeon  who  was  called  into  a  hospital  to  operate 
on  a  patient  who  required  emergency  surgery.  This  surgeon  performed  a  successful 
surgery  which  ultimately  saved  the  patient’s  life.  The  poignant  part  of  the  story  is  that 
the  surgeon’s  father  had  just  died,  although  no  one  in  the  hospital  knew  about  the  tragedy 
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at  the  time.  Even  after  receiving  horrendous  personal  news,  the  surgeon  refused  to 
abandon  his  patient. 

Several  participants  also  commented  that  their  dedication  to  patient  care  helped 
them  work  through  the  fatigue  of  lengthy  operations.  A  sense  of  duty  to  patients  often 
compels  a  surgeon  to  spend  ample  time  with  a  patient  and  family  members  prior  to 
surgery  to  ensure  that  all  questions  and  concerns  regarding  the  disease  and  procedure  are 
addressed.  Postoperatively,  this  same  sense  of  duty  brings  a  surgeon  to  a  patient’s 
bedside  and  drives  him  or  her  to  follow  the  recovery  meticulously.  Furthermore, 
dedication  to  patient  care  often  compels  a  surgeon  to  be  willing  to  hold  the  hand  of  a 
patient  who  is  very  upset.  In  fact,  a  few  participants  admitted  to  crying  with  several  of 
their  patients  during  these  highly  emotional  times. 

Another  experience  that  several  experts  seemed  to  share  was  a  long-standing 
relationship  with  a  few  of  their  patients  who  had  extremely  critical  diseases  or  injuries. 
Some  participants  revealed  that  they  still  receive  Christmas  cards  from  patients  that  they 
operated  on  five,  ten,  or  even  fifteen  years  ago.  Many  of  these  patients  acknowledge  that 
they  would  not  be  alive  if  it  were  not  for  the  valiant  efforts  of  their  surgeons.  Moreover, 
surgeons  felt  honored  that  they  continued  to  remain  in  their  patients’  thoughts  and 
prayers  so  many  years  later.  A  handful  of  participants  admitted  becoming  attached  to 
patients  and  that  these  patients  continued  to  be  a  part  of  their  lives  years  after  the  surgery. 
One  participant  recently  wrote  a  letter  of  recommendation  for  a  former  patient  who 
decided  to  apply  to  law  school.  Another  participant  shared  a  story  of  a  surgeon  who 
developed  such  a  strong  rapport  with  a  patient  that  he  found  employment  for  his  former 
patient  in  the  hospital  in  which  he  worked.  These  stories  exemplify  the  unique  bond  that 
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sometimes  exists  between  an  exceptional  surgeon  devoted  to  patient  care  and  his  or  her 
patient. 

Dedication  to  patient  care  extends  from  the  initial  meeting  with  the  patient 
through  the  surgery  and  post-operative  visits,  and  in  certain  cases,  it  may  last  for  years 
into  the  future  via  annual  cards  or  visits.  Although  it  is  impossible  to  expect  a  surgeon  to 
maintain  a  close  relationship  with  all  patients,  the  exceptional  surgeon  consistently 
provides  comfort  when  delivering  bad  news  and  sits  down  with  a  patient  and  his  or  her 
family  to  answer  all  of  their  questions  and  quell  their  fears.  It  is  this  type  of  surgeon  who 
does  everything  in  his  or  her  power  to  ensure  the  well-being  of  a  patient. 

It  was  also  frequently  noted  that  the  exceptional  surgeon  is  forced  to  make  many 
personal  sacrifices,  including  missing  important  family  events.  However,  none  of  the 
expert  participants  lamented  the  fact  that  they  sometimes  miss  these  occasions.  There 
appeared  to  be  an  unspoken  understanding  that  personal  sacrifices  are  a  natural 
consequence  of  being  in  such  a  demanding,  patient-driven  profession.  In  fact,  nearly 
every  participant  spoke  about  the  profession  with  passion  and  enthusiasm. 

Competency  #2:  Integrity 

Nineteen  expert  participants  indicated  that  integrity  was  an  essential  characteristic 
of  the  exceptional  surgeon.  A  total  of  33  comments  were  offered  concerning  the 
importance  of  integrity  for  surgery.  Specifically,  integrity  involves  the  recognition  of 
one’s  strengths  and  weaknesses  as  a  surgeon.  It  also  includes  the  willingness  to  admit 
when  one  is  wrong  and  the  public  acceptance  of  responsibility  for  mistakes  made  either 
in  or  out  of  the  OR.  In  addition,  integrity  helps  guide  a  surgeon’s  decision  making  when 
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considering  whether  to  operate  on  a  patient.  Clearly,  if  a  surgeon  performs  an  operation 
that  is  not  needed,  his  or  her  integrity  would  be  called  into  question. 

Not  every  surgeon  has  the  ability  or  the  inclination  to  communicate  honestly  and 
directly.  According  to  the  experts,  an  exceptional  surgeon  communicates  with  everyone 
in  an  honest  and  straightforward  manner,  including  patients,  family  members,  colleagues, 
residents,  and  students.  A  surgeon  with  integrity  can  be  trusted  to  inform  patients  of  his 
or  her  limitations  and  not  provide  them  with  false  hope  or  misinformation  about  their 
prognosis.  Likewise,  if  a  surgeon  has  limited  experience  with  a  particular  procedure,  it  is 
expected  that  he  or  she  will  provide  the  patient  with  an  honest  representation  of  his  or  her 
skills  and  experience.  A  surgeon  with  integrity  will  be  candid  with  the  patient  about 
whether  he  or  she  can  help. 

Exceptional  surgeons  have  a  clear  understanding  of  right  and  wrong  and  a  set  of 
ethical  principles  that  guide  them  in  their  practice.  Many  of  the  expert  participants 
believed  strongly  that  a  surgeon  should  not  perform  a  surgery  unless  the  patient’s  full 
medical  profile  had  been  carefully  considered.  Certain  patients  are  not  good  candidates 
for  surgery.  Therefore,  if  a  procedure  will  put  a  patient  at  considerable  risk,  a  surgeon 
may  have  to  refuse  to  perform  it.  One  participant  commented  that  he  must  occasionally 
refuse  older  patients  because  he  feels  it  is  not  in  their  best  interest  to  have  surgery.  He 
offers  them  alternatives  to  surgery  instead. 

Additionally,  a  surgeon  with  integrity  will  be  straightforward  with  patients  and 
their  family  members,  as  well  as  colleagues  when  mistakes  are  made.  Several 
participants  described  similar  accounts  of  surgeons  who  accepted  the  blame  for  mistakes 
that  occurred  in  the  OR,  even  when  other  surgeons  were  directly  responsible.  For 
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example,  one  participant  recalled  a  time  in  his  training  when  an  attending  surgeon 
assumed  full  responsibility  for  a  complication  caused  by  his  resident.  Furthermore, 
surgeons  with  integrity  do  not  allow  mistakes  to  be  minimized  or  ignored.  Rather,  they 
take  responsibility  for  them  and  candidly  explain  the  nature  of  the  mistake  or 
complication  to  patients,  family  members,  and  to  colleagues  at  morbidity  and  mortality 
meetings. 

Competency  #3:  Tireless  Work  Ethic 

Sixteen  participants  indicated  that  an  exceptional  surgeon  has  a  tireless  work 
ethic.  In  fact,  29  comments  were  offered  regarding  this  competency.  For  a  surgeon,  a 
tireless  work  ethic  includes  limitless  energy  when  dealing  with  patients,  family  members, 
residents,  and  colleagues.  Underlying  this  competency  is  a  passion  for  surgery  that 
drives  an  individual  to  work  through  the  exhaustion  inherent  in  a  surgeon’s  schedule. 
Exceptional  surgeons  will  persist  and  push  through  procedure  after  procedure  due  to  their 
passion  for  hard  work  and  excellence. 

Several  experts  argued  that  a  tireless  work  ethic  is  not  limited  to  the  early  years  of 
practice.  Rather,  it  is  a  goal-oriented  behavior  pattern  that  continues  throughout  a 
surgeon’s  career.  Surgeons  with  a  tireless  work  ethic  are  self-motivated  and  have  an 
intense  desire  to  succeed.  However,  success  is  not  necessarily  a  byproduct  of  hard  work 
but  a  process  of  achievement.  Exceptional  surgeons  derive  enormous  satisfaction  in  the 
process  of  performing  surgery.  They  also  find  excitement  in  rising  to  a  challenge,  such 
as  successfully  completing  a  complicated  or  life-threatening  surgery.  Moreover,  the 
exceptional  surgeon  is  one  who  successfully  engenders  the  same  enthusiasm  and  drive  for 
hard  work  in  others,  including  residents  and  colleagues. 
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Finally,  a  strong  work  ethic  paired  with  high  energy  often  results  in  a  tenacious, 
persistent  surgeon.  Five  participants  remarked  that  surgeons  with  a  tireless  work  ethic 
are  often  difficult  to  pull  off  task.  These  individuals  rarely  let  themselves  give  up  when 
dealing  with  a  problem.  They  will  devote  extremely  long  hours  to  find  solutions  to 
problems.  Several  participants  also  indicated  that  an  unresolved  problem  will  keep  them 
up  at  night.  These  surgeons  are  committed  to  working  any  time  of  day  to  tackle  a 
problem. 

Competency  #4:  Preparedness 

Fourteen  participants  viewed  preparedness  as  a  central  competency  for  surgeons. 
Twenty-seven  participant  comments  indicated  that  preparedness  distinguishes  the 
exceptional  from  the  typical  surgeon.  In  the  context  of  surgery,  preparedness  refers  to 
how  a  surgeon  approaches  his  or  her  cases.  The  prepared  surgeon  acts  thoughtfully  and 
precisely  so  that  complications  are  anticipated  prior  to  surgery  and  contingencies  have 
been  considered  should  any  complication  arise. 

Furthermore,  a  prepared  surgeon  conducts  every  operation  in  a  structured  manner 
so  that  residents  and  other  team  members  can  easily  anticipate  and  assist  with  each  step 
of  the  procedure.  An  organized  and  structured  style  of  work  also  decreases  the  chances 
that  a  surgeon  will  panic  or  lose  focus  should  unexpected  problems  arise.  Although 
complications  do  occur  even  with  the  most  prepared  surgeons,  several  participants 
commented  that  truly  prepared  surgeons  rarely  find  themselves  in  major  trouble  and  they 
tend  to  have  the  best  outcomes  with  their  patients. 

In  addition  to  performing  well-planned  and  well-executed  surgeries,  the  prepared 
surgeon  also  pays  great  attention  to  detail.  The  ability  to  attend  to  small  details  enables  a 
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surgeon  to  identify  minor  problems  and  prevent  them  from  quickly  spiraling  into  major 
problems  or  complications.  Moreover,  prepared  surgeons  are  often  viewed  by  others  as 
compulsive,  thorough,  and  orderly.  One  expert  participant  even  described  himself  as  a 
“control  freak.”  Although  individuals  who  are  compulsive  and  controlling  may  be 
viewed  somewhat  negatively  in  other  professions,  these  characteristics  appear  to  be 
strongly  linked  to  success  in  surgery. 

Competency  #5:  Intellectual  Giftedness  and  Curiosity 

Ten  participants  indicated  that  a  combination  of  intellectual  giftedness  and 
intellectual  curiosity  distinguishes  expert  from  average  surgeons.  Twenty-three 
comments  addressed  intellectual  giftedness  and  curiosity.  Surgeons  with  this 
competency  have  the  ability  to  process  an  enormous  amount  of  medical  information  and 
grasp  new  concepts  very  quickly.  They  are  also  extremely  active  in  research  and  often  in 
the  development  of  improved  surgical  procedures.  Involvement  in  these  activities 
demonstrates  curiosity  and  an  ability  to  advance  the  field  through  innovative  thinking. 

Not  only  do  exceptional  surgeons  have  an  extensive  medical  knowledge  base, 
they  can  process  the  information  that  they  need  very  quickly.  They  also  know  where  to 
find  the  information  they  need  in  a  pinch.  When  new  concepts  or  surgical  methods  arise, 
these  surgeons  are  likely  to  master  them  rapidly.  One  reason  why  they  may  grasp  new 
concepts  quickly  is  that  their  intellectual  giftedness  and  curiosity  compels  them  to 
participate  in  research.  Several  participants  indicated  that  it  is  in  the  exceptional 
surgeon’s  nature  to  ask  questions  about  the  field  and  to  strive  toward  moving  the  field 
forward  through  the  development  of  innovative  procedures.  Finally,  while  some 
surgeons  may  be  reluctant  to  tackle  critical  medical  issues,  exceptional  surgeons  tend  to 
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derive  enormous  satisfaction  from  handling  a  crisis.  Rather  than  be  held  back  by  fear  of 
failure,  an  intellectually  gifted/curious  surgeon  welcomes  a  challenge. 

Competency  #6:  Humility 

The  next  highest-ranking  competency  was  humility,  which  was  endorsed  by  eight 
participants.  A  total  of  21  comments  related  to  humility  were  provided  by  the 
participants.  Humility  reflects  how  surgeons  relate  to  other  professionals  who  are  lower 
in  rank  (e.g.,  nurses,  residents),  as  well  as  how  they  interact  with  patients  and  their  family 
members.  Humility  is  a  characteristic  often  lacking  in  the  stereotypical  version  of  a 
surgeon.  Nonetheless,  a  surgeon  with  humility  allows  patients  to  feel  comfortable 
enough  to  ask  questions  and  voice  concerns.  It  also  enables  healthcare  professionals  who 
work  alongside  the  surgeon  to  feel  respected  and  valued  for  their  contribution  to  patient 
care. 

Surgeons  with  humility  demonstrate  modesty  when  dealing  with  patients  and 
colleagues  yet  they  retain  their  self-confidence.  Remaining  humble  is  often  easier  said 
than  done  for  surgeons  given  that  they  are  the  team  leaders  in  the  OR.  Although 
surgeons  are  generally  revered  by  the  public,  exceptional  surgeons  do  not  act  as  if  they 
are  above  others.  The  exceptional  surgeon  is  humble  rather  than  arrogant  and  confident 
rather  than  cocky. 

Another  aspect  of  humility  concerns  the  display  of  respect  for  other  healthcare 
professionals,  regardless  of  their  rank  in  the  hospital  or  academic  institution.  In  other 
words,  surgeons  with  humility  do  not  talk  down  to  nurses  or  technicians,  nor  do  they 
patronize  those  who  are  beginning  their  training.  Several  participants  stressed  that 
exceptional  surgeons  are  respectful  in  all  of  their  professional  encounters.  When 
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surgeons  fail  to  treat  others  with  respect,  they  are  essentially  devaluing  their  contributions 
to  patient  care.  An  exceptional  surgeon  acknowledges  the  teamwork  inherent  in  surgery 
and  the  value  in  each  part  played  by  every  team  member. 

Competency  #7:  Compassion 

Eleven  participants  indicated  that  compassion  also  distinguishes  expert  from 
average  surgeons.  Nineteen  expert  comments  addressed  this  competency.  Compassionate 
surgeons  display  a  caring  attitude  toward  co-workers  and  residents.  These  surgeons  are 
genuinely  sympathetic  toward  patients  who  are  suffering.  They  also  display  empathy 
with  residents  who  may  be  struggling  to  perform  a  procedure.  Compassionate  surgeons 
avoid  turning  surgery  into  a  purely  technical  feat.  Rather,  they  interact  with  their 
patients,  residents,  and  colleagues  with  kindness  and  patience. 

Surgeons  with  compassion  give  other  individuals,  including  patients  and 
residents,  sufficient  time  to  talk.  Compassionate  surgeons  never  rush  patients  out  of  their 
office,  particularly  when  they  have  to  deliver  bad  news.  They  take  the  time  to  listen  to 
their  patients’  problems.  Occasionally,  these  surgeons  form  bonds  with  their  patients, 
especially  those  with  chronic  conditions.  One  participant  noted  that  he  has  actually 
attended  several  of  his  patients’  funeral  services.  Another  participant  spoke  of  his  mentor 
who  was  extremely  compassionate.  Not  only  was  he  kind  toward  his  patients,  but  he  was 
described  as  a  father  figure  with  the  residents.  His  residents  knew  they  could  call  him  at 
any  time.  In  fact,  he  even  responded  to  a  late  night  phone  call  to  bail  a  resident  out  of  a 
jail.  Not  every  surgeon  has  the  personality  to  be  overtly  compassionate,  but  exceptional 
surgeons  tend  to  exhibit  a  uniquely  sympathetic  attitude  toward  others. 
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Competency  #8:  Devotion  to  the  Field 

Eight  expert  participants  indicated  that  a  strong  devotion  to  the  field  was  an 
important  component  of  an  exceptional  surgeon.  A  total  of  16  comments  comprised  this 
competency.  In  describing  exceptional  surgeons,  some  of  the  words  the  participants  used 
were  “passionate,  “single-minded”,  and  “fanatical.”  These  descriptions  suggest  that 
many  exceptional  surgeons  display  an  intense  dedication  to  their  field  above  and  beyond 
that  of  other  surgeons.  Colleagues  and  co-workers  of  exceptional  surgeons  recognize  the 
passion  they  hold  for  surgery.  It  is  something  they  typically  wear  on  their  sleeves. 

One  participant  described  this  extreme  form  of  commitment  as  “living  and 
breathing  surgery.”  Another  participant  stated  that  this  type  of  surgeon  “gives  his  life 
and  soul  to  the  field.”  These  individuals  tend  to  put  in  more  time  on  the  job  than  other 
surgeons.  In  fact,  to  these  select  few,  surgery  is  much  more  than  a  job,  it  is  a  calling. 
Surgeons  who  are  truly  devoted  to  the  field  are  not  in  the  profession  for  money.  Rather, 
they  are  in  it  for  the  love  of  helping  and  healing  others.  These  surgeons  never  watch  the 
clock  or  espouse  a  work  shift  mentality.  In  this  vein,  one  participant  compared  himself 
to  an  artist  because,  as  he  stated,  “an  artist  never  watches  the  clock;  he  leaves  the  studio 
only  when  he  feels  he  is  finished.”  Likewise,  devoted  surgeons  remain  on  the  job  until 
they  feel  confident  that  their  work  is  truly  completed. 

Competency  #9:  Rapid  Decision  Making 

Although  rapid  decision  making  received  only  1 5  comments  from  experts,  ten 
participants  felt  compelled  to  describe  it  as  an  important  characteristic  that  distinguishes 
the  exceptional  surgeon.  There  are  two  important  aspects  of  this  competency.  First,  the 
exceptional  surgeon  often  possesses  the  ability  to  swiftly  analyze  the  risks  and  benefits  of 
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multiple  courses  of  treatment  and  choose  the  most  appropriate  and  expeditious  path  to  a 
positive  outcome.  Second,  these  surgeons  are  frequently  able  to  make  informed  decisions 
under  highly  stressful  situations  with  incomplete  data. 

Rapid  decision  making  reflects  the  ability  to  improvise  and  think  quickly  on  one’s 
feet.  Several  participants  indicated  that  surgeons  with  rapid  decision  making  ability  can 
very  quickly  size  up  a  disease  and  the  potential  complications  associated  with  it.  These 
individuals  rarely  hesitate  or  second-guess  themselves.  They  are  able  to  remain  calm  so 
they  can  concentrate  fully  on  the  proper  course  of  action.  One  particular  participant 
described  rapid  decision  making  as  the  ability  to  “maneuver  through  mine  fields.”  These 
exceptional  surgeons  know  best  how  to  handle  the  most  dangerous  and  complicated  of 
situations. 

Competency  #10:  Passion  for  Teaching 

The  final  competency  gleaned  from  the  expert  comments  was  a  passion  for 
teaching.  Twelve  participants  offered  15  comments  indicating  that  a  passion  for  teaching 
distinguishes  expert  from  average  surgeons.  Exceptional  surgeons  display  a  profound 
love  for  education  and  a  tremendous  dedication  to  their  medical  students  and  residents 
both  in  and  out  of  the  classroom  and  the  OR.  One  participant  recalled  a  former  teacher 
who  would  hold  discussions  about  cases  with  his  residents  for  hours,  even  late  in  the 
evening,  despite  the  fact  that  his  family  was  waiting  for  him  at  home.  Another  surgeon 
also  reminisced  fondly  about  a  former  teacher,  who  had  an  incredibly  hectic  schedule,  but 
still  spent  hours  talking  to  him  about  cases.  This  special  type  of  surgeon  takes  surgical 
education  very  seriously. 
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Several  participants  noted  that  surgeons  with  a  passion  for  teaching  are  also 
devoted  to  the  development  of  their  mentees.  They  enjoy  helping  residents,  fellows,  and 
even  junior  faculty  members  in  their  professional  development.  In  fact,  they  often  do 
whatever  is  in  their  power  to  facilitate  the  success  of  others.  For  this  type  of  surgeon, 
there  is  a  thrill  in  motivating  someone  else  to  achieve.  The  passionate  educator  often 
maintains  a  sense  of  responsibility,  and  certainly  an  enthusiasm,  for  creating  the  next 
generation  of  highly  qualified  surgeons. 

Characteristics  of  a  Poor  Surgeon 

The  comment  analysis  of  the  characteristics  of  a  poor  surgeon  consisted  of  62 
individual  comments  made  by  the  expert  surgeons.  Each  comment  was  placed  into  five 
overall  themes.  However,  the  fifth  theme  consisted  of  only  five  comments;  therefore,  it 
was  not  considered  to  be  representative  of  the  participant  pool.  The  following  section 
describes  the  four  predominant  characteristics  of  a  poor  surgeon.  The  most  frequently 
noted  characteristic  consisted  of  1 8  comments  and  the  least  frequently  noted 
characteristic  consisted  of  10  comments.  Furthermore,  there  were  five  comments  that  did 
not  relate  to  any  of  the  established  themes  and  thus  were  not  included  in  the  analysis  (See 
Appendix  B). 

Characteristic  #1:  Dishonesty 

A  total  of  1 8  comments  revealed  that  dishonesty  was  the  primary  characteristic  of 
a  poor  surgeon.  Dishonesty  can  manifest  itself  in  several  ways.  Individuals  may  falsify 
records  or  data,  submit  dual  publications,  or  perform  surgeries  that  are  ethically 
questionable.  A  dishonest  surgeon  may  also  fail  to  take  ownership  of  his  or  her  mistakes. 
Seven  participants  commented  that  dishonest  surgeons  often  refuse  to  take  responsibility 
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for  poor  patient  outcomes.  One  participant  discussed  an  incident  involving  a  technically 
weak  surgeon  who  had  a  poor  patient  outcome.  This  surgeon  blamed  the  nurses  for  the 
outcome  and  responded  very  poorly  to  the  suggestion  that  the  patient  should  go  back  into 
the  OR.  Dishonest  surgeons  tend  to  lie  about  their  patients’  complications  and  display  an 
unwillingness  to  accept  responsibility  for  their  errors. 

In  an  academic  setting,  a  dishonest  surgeon  may  falsify  data  or  alter  the  results  of 
a  research  study.  One  participant  recalled  a  junior  surgeon  in  his  lab  who  misrepresented 
the  data  he  collected.  As  a  result,  the  participant  could  never  trust  the  junior  surgeon 
again.  Another  participant  discussed  the  case  of  an  academic  surgeon  who  submitted  the 
same  abstract  to  two  different  conferences.  In  both  of  these  cases,  the  surgeons’ 
dishonest  actions  seriously  damaged  their  reputations. 

Five  participants  described  dishonest  surgeons  as  those  who  either  perform 
procedures  on  patients  who  do  not  need  them,  or  perform  procedures  for  which  they  are 
unqualified.  The  participants  offered  several  reasons  why  a  surgeon  might  perform  an 
unnecessary  surgery.  Some  surgeons  are  driven  by  greed  and  may  perform  unnecessary 
surgeries  for  their  own  financial  gain.  Other  surgeons  attempt  to  establish  themselves  as 
experts  by  conducting  surgeries  that  are  extremely  complicated  but  may  not  be  in  the  best 
interest  of  the  patients.  These  surgeons  may  become  fanatical  about  performing  as  many 
surgeries  as  possible  so  that  they  will  receive  additional  referrals  from  other  physicians. 
For  example,  one  participant  described  a  surgeon  with  whom  she  worked.  This  surgeon 
was  gifted  technically,  but  almost  amoral  in  practice  and  became  consumed  with  doing  as 
many  cases  as  he  could  and  getting  as  many  referrals  as  possible.  Furthermore,  he 
demonstrated  minimal  concern  for  his  patients  once  they  were  out  of  surgery.  This 
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surgeon  exemplifies  the  prototypical  dishonest  surgeon  whose  need  for  recognition, 
success,  or  money  overshadows  his  dedication  to  patient  care. 

Characteristic  #2:  Apathy  Toward  Patient  Care 

Thirteen  individual  comments  indicated  that  apathy  toward  patient  care  is 
characteristic  of  a  poor  surgeon.  Surgeons  with  this  characteristic  exhibit  a  lack  of  caring 
and  commitment  toward  their  patients.  Two  participants  stated  that  these  surgeons  also 
display  little  enthusiasm  and  energy  when  dealing  with  patients  and  their  family 
members. 

Several  participants  shared  stories  of  surgeons  who  fit  this  mold.  One  extreme 
example  involved  a  cardiac  surgeon  who  delegated  too  much  responsibility  to  the  nurses 
and  junior  surgeons.  The  surgeon  actually  left  in  the  middle  of  a  procedure,  instructing 
the  junior  surgeon  to  handle  it.  Tragically,  the  patient  died  during  the  procedure.  When 
the  junior  surgeon  called  the  senior  surgeon  to  inform  him  that  the  family  was  inquiring 
about  what  went  wrong,  he  refused  to  return  to  the  hospital  to  talk  to  the  family.  This 
surgeon  demonstrated  complete  apathy  toward  his  patients. 

Another  participant  provided  a  more  typical  example  of  a  surgeon  who  scheduled 
patients  around  his  golf  games.  He  also  had  very  limited  office  hours  and  refused  to 
extend  them  for  any  reason.  His  schedule,  whether  it  involved  golf  or  other  personal 
matters,  consistently  took  priority  over  his  professional  commitments.  Both  of  these 
examples  describe  surgeons  who  are  not  committed  to  their  patients  and  do  not  have  their 
patients’  best  interests  in  mind. 
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Characteristic  #3 :  Aggressive  Behavior 

Twelve  comments  indicated  that  poor  surgeons  often  bully  other  people  and 
behave  aggressively,  particularly  in  the  OR.  The  recipients  are  typically  residents  and 
nurses,  but  they  can  be  anyone  working  alongside  surgeons  in  the  OR.  All  12  comments 
described  intimidating  surgeons  who  often  resorted  to  yelling,  using  profanity,  and 
publicly  berating  or  demeaning  co-workers.  One  participant  recalled  an  episode  in  which 
a  surgeon  actually  became  violent  in  the  OR.  When  this  individual  was  training  a  junior 
surgeon,  he  punched  his  subordinate  for  making  a  mistake  during  the  procedure.  Not 
only  did  the  senior  surgeon  overreact,  but  he  behaved  in  an  inappropriate  and  abusive 
manner. 

Two  participants  discussed  former  teachers  who  commonly  used  public 
humiliation  as  a  teaching  strategy.  One  of  these  teachers  would  frequently  mock 
residents  who  were  beginning  their  training,  particularly  if  he  thought  they  had  talent. 
Rather  than  encourage  the  residents  to  succeed,  this  surgeon  would  try  to  undermine  their 
confidence  by  ridiculing  them  as  they  worked  on  cases.  As  the  participant  stated,  this 
surgeon  was  “trying  to  keep  the  residents  in  their  place.” 

Characteristic  #4:  Lack  of  Preparation  and  Focus 

The  final  characteristic  of  a  poor  surgeon  is  a  lack  of  preparation  and  focus, 
particularly  in  the  OR.  Ten  comments  indicated  that  a  disorganized  OR  is  a  sure  sign  of 
an  unprepared  and  unfocused  surgeon.  Although  such  surgeons  may  feel  prepared  to 
perform  a  particular  surgery  because  they  have  done  many  similar  procedures  in  the  past, 
they  often  overlook  how  much  prep  work  is  involved.  They  rarely  read  relevant  articles 
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or  books  that  would  help  them  better  prepare  for  major  surgeries  or  deal  with 
unanticipated  complications. 

Two  participants  remarked  that  unprepared  surgeons  are  often  hesitant  and  self- 
doubting.  They  are  indecisive  when  important  decisions  need  to  be  made  (e.g.,  “Should  I 
replace  or  repair  the  valve?”).  According  to  the  participants,  unprepared  surgeons  are 
frequently  “treading  water”  in  that  they  waste  valuable  time  debating  their  next  move. 
Their  operations  tend  to  take  longer  than  those  of  surgeons  who  are  prepared.  Moreover, 
these  surgeons  never  seem  to  salvage  a  marginal  patient.  Although  they  may  do  well  on 
cases  without  complications  and  patients  who  are  relatively  healthy,  their  outcomes  tend 
to  be  less  favorable  with  more  vulnerable  patients. 

Finally,  surgeons  who  lack  preparation  and  focus  may  not  attend  to  smaller, 
technical  details,  such  as  blood  loss.  They  may  have  to  rely  on  their  team  members  to 
keep  them  abreast  of  subtle  changes.  These  surgeons  may  also  omit  important  tests  that 
should  be  run  prior  to  sending  a  patient  for  surgery.  In  addition,  they  may  not  consider 
the  full  array  of  risk  factors  that  are  involved  in  each  individual  case.  When  a  surgeon 
overlooks  important  factors  prior  to  surgery,  disastrous  outcomes  may  result.  For 
instance,  one  participant  described  a  surgeon  who  had  a  young  patient  bleed  to  death 
immediately  following  a  splenectomy  procedure  (i.e.,  removal  of  the  spleen)  because  the 
surgeon  did  not  realize  she  was  on  a  blood  thinner.  This  tragedy  could  have  been 
avoided  if  the  surgeon  had  reviewed  his  patient’s  medication  record  prior  to  performing 


surgery. 
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Characteristics  of  an  Exceptional  Surgeon  of  the  Future 
The  analysis  for  an  exceptional  surgeon  of  the  future  consisted  of  101  comments. 
Participants  were  asked  to  discuss  the  “knowledge,  skills,  and  other  characteristics” 
necessary  for  a  surgeon  to  be  exceptional  in  the  year  2015.  Four  themes  emerged.  The 
most  frequently  noted  theme  included  30  comments  and  the  least  frequently  noted  theme 
included  8  comments.  Six  other  themes  will  not  be  discussed  because  they  consisted  of 
five  or  fewer  comments.  Further,  there  were  seven  comments  that  did  not  relate  to  any  of 
the  established  themes  and  thus  were  not  included  in  the  analysis  (See  Appendix  B). 
Theme  #1 :  Willingness  to  Adopt  New  Technologies  and  Skill  Sets 

Nearly  every  generation  of  surgeons  is  faced  with  the  task  of  mastering  new 
technology.  For  instance,  during  World  War  II,  surgeons  were  forced  to  learn  new 
technologies  to  deal  with  the  vast  number  of  casualties  including  the  use  of  prosthetics, 
bone  fusion,  and  the  use  of  antibiotics.  In  the  coming  decade,  surgeons  will  also  need  to 
embrace  new  forms  of  technology.  It  is  likely  that  skills  and  expertise  not  traditionally 
associated  with  surgery,  such  as  radiology,  ultrasound,  and  cathlabs,  will  become  an 
important  part  of  the  surgeon’s  arsenal. 

Thirty  expert  comments  revealed  that  surgeons  of  the  future  will  need  to  adopt 
various  technologies  and  learn  new  skill  sets  to  be  considered  exceptional.  Ten 
participants  stated  that  future  surgeons  will  need  to  embrace  MIS  procedures  and  five 
experts  indicated  that  robotics  and  computer-assisted  surgeries  will  play  a  significant  role 
throughout  the  next  decade.  Regardless  of  the  specific  type  of  technology,  it  will  be 
important  for  surgeons  to  develop  skills  that  they  might  not  have  learned  during 
residency.  In  fact,  it  is  possible  that  surgeons  will  need  to  train  themselves  on  new 
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technology  several  times  throughout  their  careers.  They  must  also  be  able  to  discern 
which  technologies  are  most  beneficial  to  their  patients.  In  addition,  exceptional 
surgeons  in  the  future  will  need  to  be  able  to  use  new  technology  that  can  reduce  the 
complexity  of  procedures.  For  example,  surgeons  may  need  to  master  technology  that 
provides  more  accurate  prognoses  and  staging  of  various  types  of  cancer. 

Surgery  has  become  less  “hands-on”  as  lasers,  robot  assistants,  and  laparoscopic 
instruments  reduce  or  eliminate  direct  contact  with  patients.  Such  new  technologies  will 
allow  for  improved  treatment,  diagnostic,  and  screening  processes.  Experts  believe  that 
this  “non-touch”  trend  will  continue  over  the  next  ten  years.  One  participant  speculated 
that  future  surgeons  will  be  able  to  assess  many  patients  without  having  to  leave  their 
homes.  For  instance,  they  will  be  able  to  access  and  read  X-rays  from  their  home  e-mail 
accounts.  Indeed,  many  technological  challenges  lie  ahead  for  future  surgeons. 

Theme  #2:  Primary  Qualities  Remain  Unchanged 

Seventeen  individual  comments  conveyed  the  notion  that  the  primary 
characteristics  of  an  exceptional  surgeon  in  the  future  will  remain  the  same  as  they  are 
today.  The  participants  did  not  expect  the  overall  profile  of  an  exceptional  surgeon  to  be 
dramatically  different  because  they  believe  that  a  surgeon’s  main  responsibility  will  still 
be  dedication  to  patient  care.  Although  the  field  is  expected  to  undergo  technological 
changes,  thirteen  participants  indicated  that  those  changes  should  not  impact  the  qualities 
of  an  exceptional  surgeon. 

The  participants  who  believed  that  the  characteristics  of  outstanding  surgeons  will 
be  the  same  in  the  future  as  they  are  today,  were  also  the  same  surgeons  who  held 
dedication  to  patient  care  as  the  most  important  competency.  These  surgeons  seemed  to 
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believe  that  the  interpersonal  relationships  surgeons  have  with  their  patients  and  the 
caring  nature  with  which  they  treat  patients  are  the  defining  characteristics  of  an  excellent 
surgeon  and  these  should  not  change  over  time. 

Theme  #3:  Adjustment  to  the  80-Hour  Work  Week 

The  issue  of  the  80-hour  work  week  restriction  for  residents  is  currently  a  hot 
topic  and  according  to  1 1  participants,  it  will  continue  to  remain  a  challenge  for  future 
surgeons.  The  participants  believe  that  a  cultural  shift  is  starting  to  take  place  among 
younger  surgeons  and  will  persist  throughout  the  next  decade.  Specifically,  they  are 
concerned  that  young  surgeons  are  adopting  a  work  shift  mentality.  One  participant 
referred  to  this  new  breed  of  surgeons  as  “clock-watchers.”  Many  participants  were 
disturbed  by  this  new  mindset  because  surgeons  have  traditionally  taken  pride  in  their 
work  schedules  which  are  not  dictated  by  set  time  frames.  One  participant  noted  that 
many  residents  now  view  surgery  as  a  job  instead  of  a  calling  and  have  come  to  expect 
limited  hours  and  more  personal  freedom  than  older  surgeons. 

According  to  the  participants,  the  adjustment  to  the  80-hour  work  week  is  also 
creating  surgeons  who  adopt  different  values.  There  is  a  danger  in  the  near  future  that 
surgeons  will  no  longer  feel  the  same  sense  of  responsibility  to  patients  as  their 
predecessors.  They  will  become  accustomed  to  handing  patients  off  to  whomever  is 
working  the  next  shift.  Three  different  surgeons  may  take  care  of  a  patient  in  the  future 
as  opposed  to  the  traditional  paradigm  in  which  a  single  surgeon  closely  monitors  a 
patient’s  entire  stay  in  the  hospital.  As  one  participant  commented,  such  disruption  to  the 
continuity  of  care  may  ultimately  be  detrimental  to  patients  if  a  surgeon  does  not  fully 
describe  a  patient’s  condition  and  status  to  the  subsequent  physicians  in  charge. 
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Furthermore,  poor  continuity  of  care  may  result  in  poorer  communication  between 
surgeons  and  patients.  The  future  surgeon  will  need  to  have  strong  communication  skills 
with  patients,  physicians,  and  other  healthcare  professionals  to  offset  the  challenges 
posed  by  the  new  80-hour  work  week. 

Theme  #4:  Surgeons  Will  Become  Highly  Specialized 

Eight  expert  comments  revealed  that  there  will  be  fewer  general  and  community 
surgeons  and  many  more  specialized  surgeons  over  the  next  ten  years.  These  participants 
indicated  that  surgeons  will  begin  gravitating  toward  more  specific  specialties.  One 
participant  commented  that  every  surgeon  will  need  to  make  a  decision  about  performing 
a  combination  of  traditional  and  minimally  invasive  procedures  or  severely  limiting  their 
practice  and  becoming  proceduralists.  These  experts  believed  that  most  surgeons  will 
choose  the  latter  option.  In  other  words,  future  surgeons  will  become  highly  skilled  at 
performing  a  smaller  number  of  related  surgeries  rather  than  compiling  a  broad  repertoire 
of  different  procedures.  Unfortunately,  patients  will  be  the  ones  to  suffer  in  the  end, 
particularly  those  who  live  far  from  a  city.  Over  the  next  decade  or  so,  it  is  likely  that 
patients  will  need  to  travel  considerable  distances  for  surgery. 

Three  participants  also  indicated  that  many  future  surgeons  are  likely  to  go  to 
great  lengths  to  minimize  the  stress  associated  with  their  practice.  They  will  work  to 
maintain  more  control  over  their  lifestyles.  Once  surgeons  become  accustomed  to 
working  only  80  hours  per  week  during  their  residency,  they  will  also  come  to  expect 
more  free  time  for  their  families  and  personal  obligations  during  their  practice.  The 
experts  speculated  that  future  surgeons  will  lean  more  toward  surgical  specialties  that 
afford  the  opportunity  to  limit  their  work  hours  while  maximizing  their  earning  potential. 
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The  exceptional  surgeons  of  the  future  will  resist  the  urge  to  pursue  careers  of 
convenience.  Rather,  they  will  maintain  professional  integrity  by  keeping  the  best 
interests  of  their  patients  in  mind  while  working  as  many  hours  as  necessary  to  ensure 
optimal  patient  care. 

Secondary  Analysis 

A  second  set  of  interviews  was  performed  with  novice  surgeons  (i.e.,  surgical 
residents).  The  novice  comment  analysis  is  divided  into  the  same  three  sections  used  for 
the  experts.  The  first  section  consists  of  the  characteristics  of  an  exceptional  surgeon 
(i.e.,  the  competencies).  The  second  section  consists  of  the  characteristics  of  a  poor 
surgeon  and  the  final  section  consists  of  the  characteristics  of  the  exceptional  surgeon  of 
the  future. 

Characteristics  of  an  Exceptional  Surgeon 
The  second  comment  analysis  consisted  of  150  individual  comments  made  by  14 
novice  surgeons.  Each  comment  was  placed  into  nine  overall  themes.  However,  an  a 
priori  decision  was  made  to  include  only  themes  that  contained  comments  from  a 
minimum  of  five  participants.  Any  theme  containing  fewer  than  five  comments  was  not 
considered  to  be  characteristic  of  the  participant  pool  because  it  represented  less  than 
one-third  of  the  total  number  of  participants  (See  Appendix  C  for  the  residual  comments). 
This  guideline  yielded  nine  competencies  of  an  exceptional  surgeon  based  on  98 
comments.  The  most  frequently  noted  competency  had  19  comments  and  the  least 
frequently  noted  competency  had  8  comments. 
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Inter-rater  Agreement 

The  same  raters  who  independently  sorted  the  expert  comments  also  sorted  the 
novice  comments.  The  Cohen’s  (1960)  kappa  value  for  this  data  set  was  0.85  with  a 
standard  error  of  .04  and  was  the  same  kappa  coefficient  found  for  the  expert  comments. 
Again,  this  is  a  very  high  level  of  inter-rater  agreement. 


Table  3 


Nine  Competencies  of  an  Exceptional  Surgeon  from  the  Novice  Surgeon 's  Perspective 

Competency _ #  of  Comments  Definition _ 

Ability  to  Communicate  in  19  Demonstrating  the  ability  to  express 

Nontechnical  Terminology  complex  medical  information  into  simple 

and  straightforward  language  that  patients 
can  understand.  Avoiding  medical  jargon 
when  educating  patients  about  diseases  or 
when  discussing  treatment  plans. 


Excellent  Bedside  Manner  17  Maintaining  an  attitude  of  sincere  concern 

and  support  of  a  patient's  psychological 
state.  Demonstrating  regard  for  the  patient- 
physician  interaction  by  taking  the  time  to 
address  a  patient's  concerns  and  questions 
and  treating  them  with  compassion  and 
respect. 


Tireless  Work  Ethic  10  Displaying  a  high  level  of  energy  and  drive 

when  dealing  with  patients,  residents,  and 
other  healthcare  professionals.  Willing  and 
eager  to  work  long  hours  to  ensure 
excellence  in  one’s  career. 


Preparedness  10  Delivering  patient  care  in  a  thoughtful, 

precise,  and  deliberate  manner.  Consistently 
anticipating  complications  before  entering 
the  OR  and  having  multiple  contingencies 
covered.  Conducting  every  operation  in  a 
structured  and  orderly  way  as  not  to  confuse 
team  members. 
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Table  3-  Continued 


Competency 

#  of  Comments 

Definition 

Rapid  Decision  Making 

9 

Demonstrating  the  capacity  to  quickly 
analyze  the  risks  and  benefits  of  multiple 
courses  of  treatment  and  to  choose  the  most 
appropriate  and  expeditious  path  to  a 
positive  patient  outcome.  Making  swift  yet 
informed  decisions  under  intense  pressure 
and  with  less  than  optimal  data  available. 

Positive  Interactions  with 
Staff 

9 

Displaying  the  ability  to  develop  a  strong 
rapport  with  all  staff  members. 

Demonstrating  respect  and  gratitude  for 
their  contribution  to  patient  care. 

Sociability 

8 

Demonstrating  a  pleasant  disposition  and  a 
good  sense  of  humor  when  interacting  with 
patients  and  hospital  employees.  Interacting 
with  others  in  a  friendly  and  outgoing 
manner. 

Maintaining  Composure 

8 

Remaining  calm  and  focused  under  highly 
stressful  situations.  Displaying  the  ability  to 
restore  order  in  the  OR  during  moments  of 
chaos  or  confusion. 

Putting  Patients  First 

8 

Displaying  total  commitment  to  patient 
care.  Consistently  willing  to  place  the  needs 
of  patients  above  personal  needs  and  family 
obligations. 
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Three  competencies  emerged  on  the  list  for  both  experts  and  novices:  Tireless 
Work  Ethic,  Preparedness  and  Rapid  Decision  Making.  It  was  decided  to  use  the  same 
definitions  for  both  sets  of  competencies  since  the  actual  comments  from  the  expert  and 
novice  participants  were  very  similar  in  content. 

Competency  #1 :  Ability  to  Communicate  in  Nontechnical  Terminology 

Nine  novice  participants  defined  the  exceptional  surgeon  as  one  who  has  the 
ability  to  communicate  with  patients  in  nontechnical  terminology.  Nineteen  comments 
reflected  the  importance  of  a  surgeon’s  ability  to  distill  complicated  medical  concepts 
into  language  that  can  be  understood  by  a  layperson.  This  competency  also  describes  a 
surgeon’s  ability  to  consider  a  patient’s  education  and  socioeconomic  status  when 
communicating  so  that  he  or  she  neither  confuses  nor  belittles  the  patient.  In  general,  the 
residents  believed  that  exceptional  surgeons  are  adept  at  avoiding  medical  jargon  and 
minimizing  the  use  of  overly  technical  language  when  talking  to  patients  and  family 
members  about  courses  of  treatment  or  when  educating  them  about  their  illnesses. 
Exceptional  surgeons  are  effective  communicators  who  can  easily  adjust  their  style  of 
communication  to  match  the  level  of  their  audience. 

Competency  #2:  Excellent  Bedside  Manner 

Ten  novice  participants  indicated  that  exceptional  surgeons,  by  definition,  have 
an  excellent  bedside  manner.  Seventeen  comments  echoed  the  notion  that  exceptional 
surgeons  are  genuinely  concerned  about  the  effects  of  illness  and  surgery  on  a  patient’s 
psychological  well-being.  These  surgeons  have  been  described  as  caring,  compassionate, 
and  empathetic  individuals  who  are  always  willing  to  take  the  time  needed  to  address 
their  patients’  questions  and  concerns.  One  resident  discussed  an  exceptional  surgeon 
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who  had  an  amazing  presence  and  ability  to  make  genuine  connections  with  his  patients 
and  put  them  at  ease.  According  to  these  novices,  exceptional  surgeons  are  committed  to 
supporting  both  the  physical  and  psychological  health  of  their  patients. 

Competency  #3:  Tireless  Work  Ethic 

Seven  novices  indicated  that  a  tireless  work  ethic  was  an  essential  component  of 
an  exceptional  surgeon.  Ten  comments  indicated  that  exceptional  surgeons  are  hard¬ 
working  professionals  with  an  untiring  dedication  to  the  job  and  their  patients. 

According  to  the  residents,  these  surgeons  make  themselves  available  to  patients, 
residents,  and  other  healthcare  professionals  at  any  time.  They  possess  the  physical 
stamina  and  energy  necessary  to  sustain  both  their  dedication  to  the  job  and  their 
indefatigable  work  schedule  over  the  course  of  their  careers. 

Competency  #4:  Preparedness 

Six  residents  stated  that  preparedness  was  a  distinguishing  characteristic  of 
exceptional  surgeons.  Ten  comments  reflected  the  idea  that  exceptional  surgeons 
practice  medicine  in  a  very  precise  and  deliberate  manner,  anticipating  potential 
complications  before  they  enter  the  OR  and  leaving  little  to  chance.  They  perform 
surgeries  in  a  structured  way  so  that  residents  and  other  surgical  team  members  can 
anticipate  each  step  and  do  not  become  confused.  Exceptional,  prepared  surgeons  also 
maintain  the  ability  to  focus  in  critical  situations.  As  one  resident  remarked  that  highly 
prepared  surgeons  can  disconnect  from  their  emotions  when  serious  complications  arise 
in  the  OR  and  get  their  patients  out  of  trouble. 
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Competency  #5:  Rapid  Decision  Making 

Five  novice  surgeons  offered  nine  comments  regarding  the  ability  of  exceptional 
surgeons  to  make  rapid  decisions.  An  outstanding  surgeon  can  analyze  the  risks  and 
benefits  of  multiple  courses  of  treatment  in  a  swift  fashion.  Additionally,  they  can 
quickly  choose  the  best  path  to  a  positive  outcome  even  under  highly  stressful  conditions. 
Exceptional  surgeons  are  decisive  and  adaptive  individuals  who  are  able  to  make 
effective  decisions,  often  with  less  than  optimal  data. 

Competency  #6:  Positive  Interactions  with  Staff 

Seven  novices  suggested  that  exceptional  surgeons  treat  staff  members  well.  Nine 
comments  reflected  the  notion  that  exceptional  surgeons  have  respect  for  and 
demonstrate  general  goodwill  to  the  other  healthcare  professionals  with  whom  they  work, 
including  residents,  nurses,  and  technicians.  Rather  than  demean  or  overlook  their 
contribution  to  patient  care,  exceptional  surgeons  often  display  gratitude  for  the  efforts  of 
all  healthcare  workers.  Consequently,  these  surgeons  tend  to  develop  a  good  rapport  with 
most  of  the  staff  members  they  encounter. 

Competency  #7 :  Sociability 

Six  novice  surgeons  believed  that  sociability  was  an  important  characteristic  of  an 
exceptional  surgeon.  Eight  comments  indicated  that  exceptional  surgeons  are  often 
sociable  individuals  who  maintain  a  pleasant  disposition  when  interacting  with  co¬ 
workers,  patients,  and  family  members.  These  participants  viewed  an  exceptional 
surgeon  as  having  a  friendly  and  outgoing  personality,  as  well  as  a  great  sense  of  humor. 
Such  surgeons  are  charismatic  individuals  who  get  along  very  well  with  most  people. 


77 


Competency  #8:  Maintaining  Composure 

Five  novice  surgeons  provided  eight  comments  on  the  ability  of  exceptional 
surgeons  to  maintain  composure  under  highly  stressful  circumstances.  Moreover,  these 
participants  noted  that  such  surgeons  can  be  relied  on  to  restore  order  during  stressful  or 
chaotic  times,  such  as  when  a  patient’s  bleeding  needs  to  be  controlled.  Three  of  the  five 
novices  used  the  word  “levelheaded”  to  describe  composed  surgeons.  Further,  a 
composed  surgeon  appears  unflustered  and  maintains  his  or  her  position  as  the  revered 
team  leader,  even  when  other  people  in  the  OR  are  in  a  state  of  panic. 

Competency  #9:  Putting  Patients  First 

Five  novices  stated  that  exceptional  surgeons  consistently  put  their  patients  before 
all  other  obligations.  Eight  comments  indicated  that  exceptional  surgeons  are  willing  to 
place  the  needs  of  their  patients  before  their  own  personal  needs  or  even  those  of  their 
family.  These  participants  described  exceptional  surgeons  as  selfless  and  completely 
focused  on  the  patients.  As  a  result,  such  surgeons  may  miss  important  personal  events, 
such  as  an  anniversary  dinner  or  their  child’s  soccer  match.  Thus,  exceptional  surgeons 
accept  and  acknowledge  that  their  patients’  well-being  must  take  precedence  over 
everything  else. 

Characteristics  of  a  Poor  Surgeon 

The  analysis  for  a  poor  surgeon  consisted  of  28  comments  made  by  14  novice 
surgeons.  Each  comment  was  placed  into  six  overall  themes.  Flowever,  three  of  these 
themes  were  endorsed  by  only  two  novices;  therefore,  they  were  not  considered  to  be 
representative  of  the  participant  pool  and  they  were  excluded  from  the  analysis.  The 
most  frequently  noted  characteristic  consisted  of  seven  comments  and  the  least  frequently 
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noted  characteristic  consisted  of  four  comments.  Moreover,  there  were  six  comments 
that  did  not  relate  to  any  of  the  established  themes  and  thus  were  not  included  in  the 
analysis  (See  Appendix  C). 

Characteristic  #1 :  Lack  of  Concern  for  Others’  Feelings 

Seven  comments  made  by  four  novice  participants  indicated  that  poor  surgeons 
show  a  lack  of  concern  for  their  patients’  feelings,  as  well  as  the  feelings  of  other  people 
with  whom  they  work.  These  surgeons  convey  little  appreciation  for  their  patients’ 
suffering.  Consequently,  patients  often  view  them  as  rude  and  disrespectful.  One  novice 
described  an  incident  in  which  a  surgeon  made  a  patient  cry  after  telling  him  that  his 
kidneys  were  going  to  lose  their  ability  to  function  because  he  was  “fat.”  He  then 
informed  the  patient  that  he  was  “going  to  die.”  The  surgeon  immediately  left  the  room, 
leaving  the  patient  alone  to  cry.  He  appeared  to  have  no  concern  over  how  his  statement 
affected  his  patient  or  the  manner  in  which  he  delivered  the  bad  news.  Surgeons  who 
share  this  characteristic  may  also  disparage  other  healthcare  professionals  by  speaking  to 
them  with  sarcasm  or  disrespect.  Still  other  surgeons  behave  unprofessionally  by  voicing 
contempt  for  the  opinions  or  ideas  of  other  physicians.  In  each  case,  these  were 
considered  to  be  examples  of  poor  surgeons  who  did  not  display  the  ability  to  treat 
patients  and  co-workers  with  dignity  and  consideration. 

Characteristic  #2:  Apathy  toward  Patient  Care 

Five  comments  made  by  five  different  novice  participants  indicated  that  apathy 
toward  patient  care  is  another  characteristic  of  poor  surgeons.  For  instance,  these 
surgeons  often  refuse  to  return  to  the  hospital  to  assist  a  patient  once  they  are  home. 

They  are  unwilling  to  shift  their  schedules  to  accommodate  their  patients  or  if  they  do, 
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they  are  often  resentful  of  patients  impeding  on  their  personal  time.  If  a  patient  develops 
sudden  complications,  these  surgeons  are  more  likely  to  delay  surgery  until  the  following 
day  when  they  are  on  schedule.  Furthermore,  two  of  the  five  participants  believed  that 
surgeons  who  are  apathetic  toward  patients  may  make  decisions  to  operate  without 
considering  their  patients’  level  of  comfort  or  prospect  for  survival.  Particularly  with 
older  patients,  poor  surgeons  may  perform  surgeries  that  are  not  in  the  best  interest  of 
their  patients  and  rob  them  of  dying  with  dignity.  The  novices  believe  it  is  important  to 
inform  patients  when  the  risks  of  surgery  outweigh  the  likelihood  of  a  good  recovery. 
Thus,  poor  surgeons  make  decisions  to  operate  without  weighing  the  risks  of  surgery 
against  the  benefits. 

Characteristic  #3 :  Aggressive  Behavior 

Four  comments  offered  by  four  residents  indicated  that  they  believe  poor 
surgeons  tend  to  exhibit  aggressive  behavior.  Such  surgeons  are  known  to  yell  at  nurses, 
technicians,  and  other  staff  members,  particularly  in  the  OR.  They  may  also  throw 
instruments  and  other  objects  when  they  become  frustrated  during  surgery.  In  essence, 
these  surgeons  are  quick  to  lose  their  temper  and  displace  their  aggression  onto  other 
personnel  who  may  be  doing  their  best  to  assist  them.  Accordingly,  surgeons  who 
demonstrate  aggressive  behavior  often  have  very  tense  ORs,  exacerbating  the  levels  of 
stress  among  other  team  members  worried  about  making  mistakes.  The  other  team 
members  realize  how  easy  it  is  to  trigger  surgeons  who  are  prone  to  aggression, 
especially  during  challenging  procedures  or  when  complications  suddenly  arise. 
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Characteristics  of  an  Exceptional  Surgeon  of  the  Future 
The  analysis  of  characteristics  of  an  exceptional  surgeon  of  the  future  consisted  of 
44  comments.  Novice  participants  were  also  asked  to  discuss  the  “knowledge,  skills,  and 
other  characteristics”  necessary  for  a  surgeon  to  be  exceptional  in  the  year  2015.  Only 
two  of  the  seven  themes  that  emerged  from  the  interviews  were  endorsed  by  over  one 
third  of  the  participants.  However,  three  other  themes  were  endorsed  by  three  (21%) 
participants.  Therefore,  the  two  highest-ranking  themes  will  be  described  in  detail  below 
and  the  three  themes  that  did  not  quite  reach  criteria  for  inclusion  will  be  discussed 
briefly.  The  most  frequently  noted  theme  consisted  of  16  comments  made  by  10  novices. 
The  next  highest  theme  consisted  of  four  comments  made  by  four  novices.  There  were 
six  comments  that  did  not  relate  to  any  of  the  established  themes  and  thus  were  not 
included  in  the  analysis  (See  Appendix  C). 

Theme  #1:  Willingness  to  Integrate  New  Technologies  into  One’s  Surgical  Practice 
Ten  residents  indicated  that  exceptional  surgeons  in  the  future  will  need  to 
embrace  new  technologies  and  procedures  and  integrate  them  into  their  practice.  The 
novices  specifically  mentioned  minimally  invasive  procedures  and  robotics.  Exceptional 
surgeons  will  need  to  go  beyond  keeping  up  with  the  literature;  they  will  need  to 
incorporate  the  latest  research  into  their  practice.  They  may  also  need  to  take  continuing 
education  courses  and  practice  their  new  skills  in  a  laboratory  setting  or  with  medical 
simulators  before  they  perform  these  procedures  on  their  patients. 

Exceptional  surgeons  will  need  to  be  highly  self-motivated  to  stay  abreast  of  the 
latest  technology.  They  must  be  open  to  learning  long  after  they  leave  their  residency 
programs.  According  to  the  novice  surgeons,  open  surgical  procedures  will  become  rarer 
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in  the  future.  Conditions  that  typically  require  surgery,  such  as  breast  or  ovarian  cancer, 
will  be  performed  through  MIS.  Therefore,  exceptional  surgeons  of  the  future  must  be 
well- versed  in  these  new  methods. 

Theme  #2:  Primary  Qualities  Remain  Unchanged 

Four  comments  made  by  the  novices  indicated  that  the  main  qualities  of  an 
exceptional  surgeon  of  the  future  will  be  the  same  as  they  are  for  surgeons  today. 
Although  the  majority  of  participants  commented  that  technological  advances  will  bring 
about  massive  changes  to  the  field,  they  did  not  believe  that  technology  will  alter  the 
characteristics  of  a  truly  competent  surgeon.  The  novice  surgeons  believe  that  an 
exceptional  surgeon  in  2015  will  still  need  to  possess  competencies  including  an 
Excellent  Bedside  Manner,  Rapid  Decision  Making,  a  Tireless  Work  Ethic,  Preparedness, 
and  the  ability  to  Maintain  Composure.  The  challenge,  as  they  see  it,  will  be  to  reach  this 
level  of  excellence  and  sustain  it  over  the  course  of  one’s  career. 

Other  Relevant  Themes 

The  following  three  themes  did  not  reach  criteria  for  inclusion,  but  they  were 
endorsed  by  over  20  percent  of  the  participants.  The  first  theme  concerned  strong 
computer  management  skills.  At  a  minimum,  exceptional  surgeons  must  be  fluent  with 
the  Internet  and  personal  computers.  They  should  also  be  able  to  retrieve  electronic 
medical  records  and  be  comfortable  using  wireless  devices,  such  as  a  Personal  Digital 
Assistant  (PDA)  to  keep  track  of  their  appointments. 

The  second  theme  focused  on  strong  interpersonal  skills.  These  skills  will  not 
only  foster  improved  communication  between  surgeons  and  patients,  but  better 
communication  among  surgeons  themselves.  Exceptional  surgeons  will  be  expected  to 
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speak  clearly  and  compassionately  with  their  patients.  According  to  one  participant, 
many  past  behaviors,  such  as  yelling  and  throwing  instruments,  will  no  longer  be 
accepted  of  surgeons  in  the  future.  The  novices  also  stated  that  surgeons  will  be  required 
to  communicate  more  readily  with  other  surgeons,  particularly  when  they  must  hand  off 
cases  due  to  vacation  plans  or  conference  schedules.  There  will  also  be  an  expectation 
for  surgeons  to  address  patients  and  other  physicians  with  respect  and  benevolence. 

Finally,  exceptional  surgeons  of  the  future  may  also  be  expected  to  successfully 
manage  an  interdisciplinary  surgical  team.  Three  residents  indicated  that  surgeons  will 
be  responsible  for  coordinating  an  increasingly  complex  team  of  people,  including 
members  of  the  critical  care  service,  nurse  practitioners,  physician  assistants,  and  surgical 
assistants.  Therefore,  good  surgeons  must  be  able  to  facilitate  interactions  among 
workers  while  minimizing  breakdowns  in  communication.  The  participants  also  stated 
that  exceptional  surgeons  must  embrace  diversity  in  future  teams  and  incorporate  both 
traditional  and  non-traditional  perspectives  and  talents. 

Comparison  between  Competencies  Identified  by  Experts  and  Novices 
One  objective  of  the  competency  analyses  was  to  identify  trends  or  patterns  that 
might  exist  between  the  experts  and  novices.  In  the  following  section,  similarities  and 
differences  between  both  competency  lists  are  addressed. 

Similarities  between  Expert  and  Novice  Competency  Results 

Several  similarities  existed  between  the  results  of  the  expert  and  novice 
interviews.  For  instance,  a  similar  number  of  competencies  were  identified  for  both 
groups  and  three  specific  competencies  were  mentioned  by  both  groups.  These 
competencies  were  Rapid  Decision  Making,  Preparedness,  and  Tireless  Work  Ethic. 
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Experts  and  novices  agreed  that  exceptional  surgeons  are  skilled  decision  makers  that  do 
not  waver  under  stress.  Both  groups  also  held  similar  views  regarding  the  importance  of 
anticipating  complications  and  setting  up  contingency  plans  before  entering  the  OR. 
Furthermore,  expert  and  novice  participants  agreed  that  exceptional  surgeons  are 
unstinting  individuals  who  work  through  the  exhaustion  of  the  job  and  maintain  high 
levels  of  energy  and  productivity  throughout  their  careers. 

The  experts  identified  Dedication  to  Patient  Care  as  the  highest-ranking 
competency  of  an  exceptional  surgeon.  This  competency  was  similar  to  the  novice’s 
lowest-ranking  competency,  Putting  Patients  First.  However,  the  emphasis  within  each 
competency  differed  slightly  for  both  groups.  The  novice  comments  focused  more  on  the 
personal  and  familial  sacrifices  that  exceptional  surgeons  make  for  their  patients.  By 
contrast,  experts  focused  more  on  the  welfare  of  their  patients.  In  general,  both  expert 
and  novice  comments  indicated  that  exceptional  surgeons  view  the  well-being  of  their 
patients  as  a  critical  component  of  their  practice. 

On  the  surface  it  appears  that  the  experts  identified  several  personality -related 
competencies,  such  as  Humility  and  Compassion  that  the  novices  did  not  mention.  It  can 
be  argued,  however,  that  there  is  some  overlap  between  Compassion  and  Excellent 
Bedside  Manner  reported  by  the  novices.  There  is  also  overlap  between  the  expert 
competency,  Humility,  and  the  novice  competency,  Positive  Interactions  with  Staff.  One 
important  aspect  of  Humility  concerns  respect  for  healthcare  professionals  regardless  of 
their  rank  or  title.  Although  experts  also  viewed  Humility  as  a  display  of  modesty  and 
recognizing  one’s  strengths  and  weaknesses  as  a  surgeon,  there  is  clearly  some  overlap 
between  this  competency  and  Positive  Interactions  with  Staff. 
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Differences  between  Expert  and  Novice  Competency  Results 

There  were  several  differences  between  the  experts  and  novices.  For  instance, 
several  competencies  that  emerged  from  the  novice  data  were  not  found  in  the  expert 
data,  including  Ability  to  Communicate  in  Nontechnical  Terminology  and  Maintaining 
Composure.  Ability  to  Communicate  in  Nontechnical  Terminology  was  the  highest- 
ranking  competency  for  novices,  but  it  was  not  mentioned  by  the  experts.  The  experts 
were  also  much  less  inclined  to  discuss  the  ability  of  surgeons  to  remain  calm  during 
stressful  or  critical  procedures. 

There  were  four  competencies  that  were  exclusive  to  the  expert  participants. 

First,  expert  surgeons  viewed  Integrity  as  an  integral  component  of  excellence  in  surgery. 
Many  experts  believed  that  a  surgeon’s  strength  of  character  was  directly  linked  to  his  or 
her  level  of  performance.  Second,  the  expert  comments  revealed  that  exceptional 
surgeons  have  high  Intellectual  Giftedness  and  Curiosity.  In  other  words,  they  display 
curiosity,  creativity,  and  innovation  when  dealing  with  both  academic  and  non-academic 
medical  issues.  Third,  experts  were  more  likely  to  describe  Devotion  to  the  Field  as  an 
important  competency.  They  believed  that  exceptional  surgeons  are  extremely  passionate 
about  their  work  and  dedicated  to  their  profession.  Finally,  the  experts  were  much  more 
focused  than  the  novices  on  the  link  between  surgical  excellence  and  Passion  for 
Teaching.  Only  the  experts  mentioned  the  important  role  that  surgeons  play  in  training 
the  next  generation  of  medical  students  and  residents. 

Similarities  between  Expert  and  Novice  Results  of  a  Poor  Surgeon 

Both  expert  and  novice  participants  identified  Apathy  toward  Patient  Care  as  one 
of  the  main  characteristics  of  a  poor  surgeon.  Experts  and  novices  commented  that  poor 
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surgeons  are  inflexible  and  even  indolent  when  they  are  asked  to  exert  extra  effort  on 
behalf  of  a  patient  (e.g.,  when  they  need  to  return  the  hospital  after  a  shift  is  over).  In 
addition,  both  groups  indicated  that  poor  surgeons  tend  to  display  Aggressive  Behavior. 
Differences  between  Expert  and  Novice  Results  of  a  Poor  Surgeon 

According  to  the  novice  participants,  poor  surgeons  tend  to  demonstrate  a  Lack  of 
Concern  for  Others’  Feelings.  Novices  made  more  comments  than  experts  regarding 
surgeons  who  are  rude  or  disparaging  toward  patients,  staff  members,  and  other 
physicians.  Although  the  experts  did  not  mention  this  characteristic,  they  identified  two 
other  qualities  that  the  novices  did  not  consider:  Dishonesty  and  Lack  of  Preparation  and 
Focus.  Overall,  the  expert  participants  focused  more  on  the  integrity  of  exceptional 
surgeons  and  lack  thereof  in  poor  surgeons. 

Similarities  between  Expert  and  Novice  Results  of  an  Exceptional  Surgeon  of  the  Future 
Both  experts  and  novices  commented  that  exceptional  surgeons  of  the  future  must 
be  willing  to  incorporate  new  technologies  and  procedures  into  their  practice.  Although 
experts  and  novices  agreed  that  the  acceptance  and  integration  of  technology  will  be 
paramount  to  future  surgeons,  they  also  believed  that  the  fundamental  characteristics  of 
an  exceptional  surgeon  will  transcend  changes  in  technology.  In  other  words,  both 
groups  stated  that  the  core  traits  of  an  exceptional  surgeon  of  the  future  will  remain  very 
similar  to  what  they  are  today. 

Differences  between  Expert  and  Novice  Results  of  an  Exceptional  Surgeon  of  the  Future 
There  was  more  variability  between  the  expert  and  novice  comments  regarding 
the  characteristics  of  an  exceptional  surgeon  in  the  future.  The  experts  generated  four 
themes  while  the  novices  generated  only  two.  The  experts  noted  that  exceptional 
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surgeons  in  the  future  must  resist  the  work  shift  mentality  that  may  become  associated 
with  the  80-hour  work  week  recently  instituted  in  residency  programs.  They  also 
indicated  that  future  surgeons  may  gravitate  toward  higher  degrees  of  specialization.  The 
novice  participants  did  not  mention  either  of  these  themes  in  their  interviews.  Instead, 
the  novices  discussed  that  an  exceptional  surgeon  of  the  future  will  remain  the  same  as 
they  are  for  surgeons  today.  They  also  indicated  that  future  surgeons  must  be  willing  to 
integrate  new  technologies  into  their  surgical  practice. 


87 


DISCUSSION 

The  overall  intent  of  this  study  was  to  gain  a  better  understanding  of  surgical 
excellence  by  interviewing  a  sample  of  outstanding  surgeons.  The  main  objective  of  the 
primary  analysis  was  to  develop  a  detailed  list  of  competencies  of  an  exceptional  surgeon 
as  well  as  a  list  of  characteristics  that  define  a  poor  surgeon.  The  second  objective  was  to 
develop  a  list  of  themes  that  would  describe  exceptional  surgeons  in  the  future.  In 
addition,  a  smaller  sample  of  novice  surgeons  was  interviewed  and  three  similar  lists 
were  generated  to  understand  how  these  surgeons  conceptualize  surgical  excellence.  The 
final  objective  was  to  examine  both  data  sets  for  similarities  and  differences  in  the  way 
experts  and  novices  define  excellence.  Therefore,  a  comparison  was  made  between  the 
expert  and  novice  lists  of  competencies,  as  well  as  the  lists  of  characteristics  of  a  poor 
surgeon  and  themes  impacting  future  surgeons  to  determine  if  different  cohorts  place 
varying  emphases  on  the  cardinal  traits  of  an  exceptional  surgeon. 

Primary  Analysis:  Expert  Comments 
Competencies  Expected  to  Emerge 

Based  upon  previous  research,  as  well  as  the  ACGME  competencies  used  in 
resident  training,  several  predictions  were  made  regarding  the  expert  data.  It  was 
anticipated  that  several  competencies  would  emerge  including  technical  skill  or 
proficiency,  critical  thinking,  leadership  ability,  intelligence,  and  interpersonal  skill. 
Technical  Skill 

In  the  field  of  surgery,  technical  skill  is  referred  to  by  several  names.  For 
instance,  technical  skill  may  be  called  hand-eye  coordination,  mechanical  excellence,  or 
psychomotor  ability.  In  the  present  study,  technical  skill  was  defined  specifically  as 
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proficiency  in  two  psychomotor  factors:  manual  dexterity  and  hand-eye  coordination. 
Surgeons  are  expected  to  have  the  technical  skills  to  handle  and  maneuver  their 
instruments  with  the  greatest  respect  for  tissue  and  maintain  economy  of  motion  during 
their  actions.  Thus,  the  first  prediction  was  that  technical  skill  would  emerge  from  the 
expert  data  as  an  important  competency. 

Although  seven  experts  mentioned  technical  skill  as  an  important  characteristic  of 
the  exceptional  surgeon,  it  did  not  reach  the  criteria  to  be  included  on  the  list  of 
competencies.  Therefore,  this  prediction  received  only  partial  support.  Instead,  the 
majority  of  experts  thought  that  technical  skills  would  eventually  be  mastered  by  most 
surgeons.  In  other  words,  technical  skill  is  necessary  but  not  sufficient  for  becoming  an 
outstanding  surgeon.  Most  of  the  experts  seemed  to  suggest  that  other  characteristics, 
such  as  Dedication  to  Patient  Care,  Humility,  and  Integrity,  were  more  important  for 
distinguishing  truly  talented  surgeons. 

Critical  Thinking 

As  previously  stated,  a  high  performing  surgeon  must  possess  a  combination  of 
technical  skills  and  higher-order  cognitive  processes,  such  as  critical  thinking  and 
independent  thought.  In  the  present  study,  critical  thinking  was  defined  as  mental 
processes  that  allow  a  person  to  analyze  and  evaluate  critical  medical  decisions, 
particularly  when  limited  data  are  available.  The  second  prediction  was  that  critical 
thinking  would  also  emerge  as  a  key  competency  of  exceptional  surgeons.  In  other 
words,  it  was  believed  that  the  ability  to  make  critical  decisions,  such  as  deciding  whether 
to  operate,  determining  diagnoses,  and  determining  the  level  of  risk  to  which  a  patient 
could  be  exposed  by  operating,  would  be  necessary  for  surgical  excellence. 
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The  experts  did  refer  to  critical  thinking;  however,  critical  thinking  was  subsumed 
within  two  separate  competencies:  Rapid  Decision  Making  and  Preparedness.  Experts 
indicated  that  exceptional  surgeons  make  swift,  informed  decisions  and  are  able  to 
quickly  analyze  the  risks  and  benefits  of  multiple  courses  of  treatment  for  a  patient.  The 
competency  definition  provided  for  Rapid-Decision  Making  was  very  similar  to  the  way 
in  which  critical  thinking  was  originally  conceptualized  in  the  present  study.  Although 
the  ability  to  anticipate  complications  before  they  arise  in  the  OR  was  not  hypothesized 
specifically,  it  would  certainly  be  considered  a  component  of  critical  thinking  in  surgery. 
It  is  also  likely  that  the  experts  were  referring  to  another  aspect  of  critical  thinking  when 
they  discussed  the  importance  of  Preparedness.  Surgeons  who  are  strong  critical  thinkers 
are  typically  prepared  to  handle  unexpected  problems  or  changes  in  a  patient’s  status. 
Since  it  was  anticipated  that  experts  would  mention  the  relationship  between  critical 
thinking  and  surgical  excellence,  the  emergence  of  both  Rapid  Decision  making  and 
Preparedness  as  competencies  supports  this  prediction. 

Leadership 

Surgeons  have  the  responsibility  of  leading  a  team  in  the  OR.  Since  the  skills 
necessary  to  be  a  strong  leader  often  come  with  many  years  of  experience,  it  was 
predicted  that  the  experts  would  describe  leadership  skills  as  another  important 
competency  of  exceptional  surgeons.  Although  leadership  skills  did  not  emerge  as  one  of 
the  top  ten  competencies,  it  is  possible  that  the  interview  questions  may  not  have 
prompted  the  experts  to  discuss  the  topic  of  leadership  explicitly.  Moreover,  aspects  of 
leadership  may  have  been  subsumed  in  other  competencies.  In  the  present  study, 
leadership  was  defined  by  qualities  including  the  ability  to  communicate  effectively, 
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demonstrate  integrity,  and  display  professionalism.  Therefore,  it  is  conceivable  that 
facets  of  leadership  were  included  in  Dedication  to  Patient  Care,  Integrity,  and  even 
Humility. 

The  experts  who  were  interviewed  are  clearly  leaders  in  their  field.  However, 
they  may  not  think  of  leadership  in  the  same  way  as  leaders  in  other  professions,  such  as 
business  and  the  military.  Also,  since  the  experts  tended  to  be  older  surgeons,  they  may 
not  have  been  aware  of  recent  leadership  research  emphasizing  team  dynamics  in  the  OR. 
In  fact,  several  articles  in  the  past  few  years  have  focused  on  the  importance  of  leadership 
in  surgical  teams  (e.g.,  Edmondson,  2003;  Wells,  Jr.,  2002).  Therefore,  it  is  very  likely 
that  future  generations  of  surgeons  may  aspire  to  have  some  of  the  same  leadership 
characteristics  commonly  observed  in  other  fields  such  as  business. 

Another  reason  why  the  experts  did  not  mention  leadership  more  frequently  could 
be  that  most  surgeons  do  not  have  many  opportunities  to  observe  their  colleagues  in  the 
OR.  Although  they  may  learn  about  the  outcomes  of  other  surgeons’  procedures,  they 
may  not  know  whether  they  are  strong  leaders  or  whether  they  successfully  facilitate 
collaboration  among  team  members.  Since  other  members  of  surgical  teams  (e.g., 
residents,  nurses,  and  surgical  assistants)  are  aware  of  and  affected  by  a  surgeon’s 
leadership  skills,  they  might  be  more  likely  to  mention  the  importance  of  such  skills  than 
the  surgeons  sampled  for  this  study. 

Intelligence 

It  was  also  predicted  that  the  experts  would  mention  the  importance  of 
intelligence  in  surgery.  For  this  study,  intelligence  was  described  as  a  combination  of 
analytical,  creative,  and  practical  thinking  skills  and  all  three  facets  were  expected  to  be 
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mentioned  by  the  experts.  The  results  revealed  that  experts  viewed  analytical  and 
creative  intelligence  to  be  the  most  relevant  forms  of  intelligence  for  exceptional 
surgeons.  In  fact,  Intellectual  Giftedness  and  Curiosity  was  ranked  fifth  on  the 
competency  list.  Experts  believed  that  exceptional  surgeons  tend  to  grasp  new  concepts 
very  quickly  and  remain  inquisitive  about  the  field  throughout  their  career.  The  ability  to 
grasp  new  concepts  quickly  reflects  analytical  intelligence.  The  experts  also  mentioned 
that  Intellectual  Giftedness  and  Curiosity  involves  the  ability  to  tackle  medical  issues  in 
an  innovative  manner.  Clearly,  this  ability  is  directly  related  to  creative  intelligence. 

Practical  intelligence,  which  is  most  often  viewed  as  “street  smarts,”  “common 
sense,”  or  intelligence  that  functions  in  the  real  world,  was  not  discussed  explicitly. 
However,  since  practical  intelligence  refers  to  the  application  of  learned  information,  it  is 
likely  subsumed  within  several  of  the  competencies,  including  Rapid  Decision  making 
and  Preparedness.  For  example,  the  ability  to  make  swift  and  informed  decisions  is 
largely  contingent  on  years  of  experience  that  contribute  to  common  sense,  as  well  as  the 
knowledge  necessary  to  solve  practical  problems.  Given  the  frequency  with  which  most 
experts  discussed  intellectual  ability  in  general,  this  prediction  was  fully  supported. 
Dedication  to  Patient  Care  and  Compassion 

As  noted  earlier,  several  researchers  have  offered  descriptions  of  a  high 
performing  surgeon.  For  example,  Cuschieri,  Francis,  Crosby,  and  Hanna  (2001)  noted 
that  outstanding  surgeons  tend  to  possess  an  amalgam  of  skills  and  abilities  including, 
clinical  skills,  technical  ability,  academic  potential,  administrative  skills,  good  judgment, 
and  the  ability  to  maintain  strong  relationships  with  patients  and  other  health  care 
professionals.  Although  the  present  competency  model  did  not  include  technical  ability 
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or  administrative  skills,  it  did  account  for  the  other  characteristics.  Specifically, 
Dedication  to  Patient  Care  and  Compassion  are  directly  related  to  clinical  skills  and  the 
ability  to  relate  well  to  patients  and  health  care  professionals.  Dedication  to  Patient  Care, 
which  emerged  as  the  highest  ranking  competency,  includes  an  unwavering  willingness 
to  address  all  of  a  patient’s  questions  and  fears.  Compassion,  which  ranked  seventh  on 
the  list,  relates  to  the  caring  and  empathetic  manner  in  which  excellent  surgeons  relate  to 
their  patients.  Because  two  competencies  concerning  interpersonal  skills  emerged,  the 
prediction  was  also  fully  supported. 

Unanticipated  Competencies  that  Emerged 

Five  competencies  emerged  on  the  expert  list  that  were  not  anticipated:  Integrity, 
Tireless  Work  Ethic,  Humility,  Devotion  to  Field,  and  Passion  for  Teaching.  Possible 
explanations  for  why  the  experts  deemed  these  competencies  to  be  important  are 
considered  below. 

Integrity 

Integrity  was  the  second  highest-ranked  competency  on  the  list.  Integrity  is  a 
characteristic  that  is  valued  in  nearly  every  profession.  The  experts  seemed  to  view 
Integrity  as  particularly  important  to  surgery  because  an  inability  to  admit  to  one’s 
weaknesses  or  accept  responsibility  for  one’s  mistakes  puts  patients  at  risk.  Several 
experts  mentioned  having  fired  surgeons  for  falsifying  data  or  records  or  for  shirking 
their  responsibility  when  complications  arose.  Since  surgeons  hold  a  position  of  great 
esteem  in  the  hospital  and  in  society,  their  opinions  and  medical  orders  often  go 
unquestioned.  According  to  many  of  the  experts,  surgeons  who  appreciate  and  respect 
the  power  they  hold  and  consistently  act  with  integrity  may  be  less  common  than  most 
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people  realize.  Therefore,  the  experts  may  have  included  Integrity  as  an  important 
competency  not  only  because  it  is  closely  tied  to  patient  safety,  but  because  it  is  difficult 
to  find  a  surgeon  who  consistently  acts  with  integrity. 

Tireless  Work  Ethic 

Similar  to  Integrity,  a  high  level  of  energy  and  drive  is  related  to  excellence 
across  many  professions.  Given  that  most  surgeons  have  long  and  exhausting  work 
schedules,  those  who  work  through  the  fatigue  and  remain  motivated  and  productive 
distinguish  themselves  from  the  majority  of  surgeons.  It  seems  inevitable  that  most 
surgeons  would  reach  a  point  in  their  careers  where  they  could  potentially  bum  out  if 
they  did  not  reduce  their  hours.  On  the  other  hand,  there  is  a  small  percentage  of 
surgeons  who  enjoy  their  work  to  such  a  degree  that  they  maintain  a  high  level  of  drive 
throughout  the  career.  These  are  the  surgeons  who  the  experts  believe  are  tmly 
exceptional. 

Humility 

Humility  is  another  characteristic  that  was  not  anticipated.  Nonetheless,  many 
experts  believe  that  those  surgeons  who  display  modesty  when  interacting  with  patients 
and  colleagues  are  often  viewed  as  excellent  surgeons.  Although  the  stereotypical  view 
of  a  surgeon’s  personality  is  often  one  of  arrogance,  the  experts  indicated  that  outstanding 
surgeons  actually  display  the  opposite  demeanor.  Once  again,  it  is  conceivable  that  the 
powerful  position  held  by  surgeons  in  hospitals  may  make  them  susceptible  to  developing 
a  “God  complex.”  The  experts  seemed  to  feel  that  great  surgeons  keep  their  ego  in  check 
and  do  not  feel  entitled  to  special  treatment.  It  does  appear  to  be  a  challenge;  however, 
for  most  surgeons  to  maintain  their  modesty  over  time.  This  may  be  the  cumulative 
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result  of  hundreds  of  patients  and  other  healthcare  professionals  treating  them  with 
deference.  Regardless  of  reasons  why  many  surgeons  might  lose  their  modesty,  it  is  not 
surprising  that  the  experts  would  deem  Humility  to  be  indicative  of  surgical  excellence. 
Devotion  to  Field 

Devotion  to  Field  was  another  unanticipated  competency  that  emerged  on  the 
expert  competency  list.  Many  experts  indicated  that  exceptional  surgeons  differ  from 
average  ones  in  the  degree  to  which  they  are  devoted  to  the  profession.  The  experts 
noted  that  a  small  minority  of  surgeons  are  so  passionate  about  their  work  that  they 
believe  it  is  their  calling  in  life.  A  few  experts  added  that  some  devoted  surgeons  have  no 
intention  of  retiring  and  actually  become  depressed  if  they  are  forced  into  retirement.  It 
would  seem  that  this  level  of  devotion  would  easily  distinguish  exceptional  surgeons 
from  the  rest. 

Passion  for  Teaching 

The  last  competency  mentioned  by  the  experts  that  was  not  expected  at  the  outset, 
was  Passion  for  Teaching.  Although  it  was  the  lowest-ranking  competency,  many 
experts  indicated  that  exceptional  surgeons  display  a  love  for  education  that  is 
disproportionate  to  most  other  surgeons.  Because  the  majority  of  experts  in  this  sample 
were  academic  surgeons,  it  seems  logical  that  they  would  focus  on  a  surgeon’s  dedication 
to  teaching  others.  Although  this  competency  may  not  apply  to  surgeons  working  in 
private  practice,  it  would  seem  to  differentiate  between  average  and  exceptional  surgeons 


in  academic  institutions. 
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Themes  Expected  to  Emerge 

In  addition  to  the  competencies,  it  was  predicted  that  three  themes  related  to 
surgery  in  the  near  future  would  be  mentioned  by  the  experts.  Specifically,  the  experts 
were  expected  to  mention  the  resident  work  hour  mandate,  new  technologies  and  skill 
sets,  and  recertification  and  professional  development.  The  interview  data  actually 
revealed  four  themes  regarding  competencies  surgeons  would  require  over  the  next  ten 
years  to  achieve  excellence  in  the  field. 

The  Resident  Work  Hour  Mandate 

As  stated  earlier,  many  experienced  surgeons  oppose  the  ACGME’s  80-hour  work 
week  limit.  In  fact,  the  Society  of  University  Surgeons  (SUS)  released  a  statement 
indicating  that  restricted  work  hours  “do  not  prepare  residents  for  the  real  world  of 
surgical  practice”  (Cole,  Bertagnolli,  &  Nussbaum,  2002).  Further,  Komenaka  (2003) 
commented  that  a  different  type  of  medical  student  may  now  be  attracted  to  the  surgical 
professional.  He  argued  that  students  who  prefer  “lifestyle  specialties”  (e.g., 
dermatology,  emergency  medicine,  anesthesia)  may  now  be  attracted  to  surgical 
programs  because  time  commitment  and  training  obligations  are  less  taxing  under  the 
new  mandate.  In  the  present  study,  the  experts  did  indeed  indicate  that  a  major  issue 
facing  surgeons  today,  and  one  that  will  continue  to  impact  them  throughout  the  next 
decade,  is  this  work  hour  restriction.  The  majority  of  experts  had  strong  opinions 
concerning  whether  this  mandate  would  be  a  benefit  or  detriment  to  the  field.  Few 
surgeons  believed  that  the  work  week  restriction  would  produce  better  surgeons.  In  fact, 
the  consensus  was  that  residents  would  be  less  prepared  than  their  predecessors  to  handle 
complicated  cases  or  work  the  long  hours  required  for  surgery.  Many  experts  also  feared 


96 


that  the  80-hour  work  week  would  eventually  decrease  to  50  or  60  hours  per  week  as  it 
has  in  Western  Europe  (Pickersgill,  2001).  It  was  generally  believed  that  exceptional 
surgeons  in  the  future  will  need  to  resist  the  work  shift  mentality  that  may  result  from  this 
restriction  if  they  wish  to  distinguish  themselves  from  other  surgeons.  Given  the 
frequency  with  which  the  experts  discussed  the  impact  of  the  ACGME  work  hour 
restriction,  the  results  of  the  present  study  support  the  statements  made  by  the  SUS  and 
Komenaka. 

New  Technologies  and  Skill  Sets 

Advancements  in  technology  are  rampant  in  nearly  every  field  of  medicine 
including  surgery.  The  experts  were  expected  to  discuss  the  impact  of  technology  on  the 
role  of  future  surgeons.  As  predicted,  most  experts  believed  that  surgeons  must  be 
prepared  to  adopt  new  technologies  and  develop  new  skill  sets  throughout  their  careers. 
Experts  indicated  that  surgeons  who  make  the  effort  to  keep  abreast  of  new  technological 
advances,  such  as  lasers  and  robot  assistants,  will  be  more  likely  than  others  to  excel  in 
the  field. 

Recertification  and  Professional  Development 

One  way  to  ensure  that  surgeons  continue  to  develop  new  skills  sets  is  through 
certification  and  recertification/re  validation  programs.  These  programs  ensure  that 
surgeons  maintain  a  high  professional  standing  and  acceptable  “qualifications  of 
practice”  (i.e.,  skills)  in  their  surgical  specialty,  as  well  as  a  commitment  to  continuing 
education  (Patil,  Cheng,  &  Wong,  2003).  Although  it  was  predicted  that  the  issue  of 
recertification  and  professional  development  would  emerge  as  an  important  theme  for 
future  surgeons,  experts  did  not  discuss  this  topic  during  their  interviews.  Since  all 
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board-certified  surgeons  must  adhere  to  the  same  guidelines,  the  certification  process 
may  not  distinguish  between  exceptional  and  typical  surgeons  in  the  future. 

Unanticipated  Themes  that  Emerged 

Stability  of  Attributes 

The  experts  mentioned  two  themes  they  believe  will  impact  future  surgeons  that 
were  not  anticipated  at  the  outset.  One  of  these  themes  concerns  the  core  attributes  of  an 
exceptional  surgeon.  Specifically,  they  indicated  that  the  core  attributes  will  not  change 
over  the  next  ten  years.  Although  the  experts  acknowledge  that  advances  in  technology 
will  always  impact  the  field,  they  argued  that  technology  does  not  characterize  excellence 
in  surgery.  Several  surgeons  stated  that  the  qualities  of  an  excellent  surgeon  are  the  same 
today  as  they  were  one  hundred  years  ago  and  are  likely  be  the  same  over  the  next 
hundred  years.  Experts  indicated  that  a  strong  commitment  toward  patients  will  always 
be  a  defining  characteristic  of  an  outstanding  surgeon.  Furthermore,  personality  traits 
such  as  Integrity,  Humility,  and  Compassion  will  be  important  regardless  of  the  era  in 
which  a  surgeon  works. 

Surgeons  Will  Become  Highly  Specialized 

The  final  theme  mentioned  by  the  experts  concerned  specialization:  future 
surgeons  are  likely  to  gravitate  toward  more  specialized  surgical  subfields.  The  experts 
mentioned  that  they  have  already  noticed  many  surgeons  limiting  their  practice  to 
perform  smaller  numbers  of  similar  procedures.  This  theme  is  related  to  the  notion  that 
the  80-work  week  restriction  is  creating  a  different  mindset  among  today’s  younger 
surgeons,  many  of  whom  are  seeking  ways  to  make  surgery  fit  their  lifestyle  rather  than 
to  adapt  their  lifestyle  to  meet  their  patients’  needs.  The  majority  of  experts  expressed 
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disdain  toward  this  new  breed  of  surgeons  who  they  described  as  lacking  commitment  to 
the  profession.  Therefore,  the  idea  that  surgeons  are  moving  away  from  being  generalists 
so  that  they  can  have  more  control  over  their  schedules  and  increase  their  income  is 
consistent  with  the  fear  experts  expressed  regarding  the  “clock-watching”  mentality  of 
the  next  generation  of  surgeons. 

Secondary  Analysis:  Novice  Comments 
Comparison  of  Competencies  Mentioned  by  Experts  versus  Novices 
The  objective  of  the  primary  analysis  was  to  use  the  expert  data  to  create  a  list  of 
competencies.  A  second  set  of  interviews  was  conducted  with  14  novice  surgeons.  The 
two  data  sets  were  compared  to  identify  patterns  that  distinguish  how  exceptional  and 
novice  surgeons  conceptualize  surgical  excellence. 

Similarities  between  Expert  and  Novice  Competency  Results 

It  was  anticipated  that  some  degree  of  overlap  would  exist  among  the 
competencies  provided  by  the  experts  and  novices,  but  on  the  whole  they  would  look 
quite  different.  That  prediction  was  confirmed.  Turning  first  to  the  few  similarities,  both 
groups  mentioned  three  specific  competencies:  Rapid  Decision  making,  Preparedness, 
and  Tireless  Work  Ethic.  The  underlying  theme  of  Rapid  Decision  Making  and 
Preparedness  is  the  ability  to  perform  at  the  top  of  one’s  game  in  the  OR  in  the  face  of 
sudden  complications  or  crises.  Experts  and  novices  alike  commented  that  exceptional 
surgeons  are  always  prepared  for  a  procedure  and  remain  vigilant  for  any  problem  that 
may  arise.  In  addition,  experts  and  novices  agreed  that  exceptional  surgeons  differ  from 
average  ones  because  experts  can  make  swift,  yet  informed  decisions  in  high  pressure 


99 


situations.  In  other  words,  such  individuals  are  not  hesitant  or  self-doubting.  They  have 
gained  the  experience  and  knowledge  to  make  good  decisions  with  confidence. 

Experts  and  novices  also  agreed  that  a  Tireless  Work  Ethic  is  another 
distinguishing  characteristic  of  exceptional  surgeons.  The  majority  of  surgeons  reach  a 
point  in  their  careers  where  the  hectic  pace  of  their  schedules  becomes  less  appealing. 
Many  surgeons  will  begin  taking  steps  to  devote  more  time  to  their  personal  lives. 
However,  both  participant  groups  believe  that  exceptional  surgeons  are  different  because 
they  have  energy  and  an  intense  drive  that  allows  them  to  continue  to  work  long  hours 
and  sacrifice  personal  time  for  their  patients.  Although  novices  were  more  likely  to  focus 
on  the  personal  and  familial  sacrifices  made  by  exceptional  surgeons  for  their  patients, 
both  groups  noted  that  outstanding  surgeons  consistently  view  their  patients  as  their 
highest  priority  throughout  their  careers.  Accordingly,  experts  and  novices  alike  believed 
that  the  drive  to  work  as  many  hours  as  necessary  for  their  patients  was  one  of  the  most 
important  characteristics  of  an  exceptional  surgeon. 

Although  the  experts  and  novices  sometimes  used  different  language  to  describe 
the  personality  of  exceptional  surgeons,  there  were  several  similar  underlying  concepts. 
For  example,  novice  surgeons  discussed  the  importance  of  a  good  bedside  manner  while 
experts  often  used  the  words  “compassion”  and  “concern”.  Both  groups  attempted  to 
define  this  trait  as  the  tendency  to  be  warm,  caring,  and  respectful  toward  patients  and 
their  family  members.  Experts  and  novices  described  exceptional  surgeons  as  truly 
connected  to  patients  and  willing  to  comfort  them,  particularly  when  bad  news  must  be 
delivered.  Moreover,  both  groups  believe  it  is  crucial  that  surgeons  treat  not  only  their 
patients  well,  but  also  other  staff  members  and  physicians.  Outstanding  surgeons 
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acknowledge  the  collaboration  inherent  in  surgical  teams  and  therefore,  never  diminish  or 
invalidate  the  efforts  of  nurses,  technicians,  and  other  individuals  who  assist  them  in  their 
efforts. 

Differences  between  Expert  and  Novice  Competency  Results 

As  expected,  there  were  more  differences  than  similarities  between  the  expert  and 
novice  competency  results.  There  were  four  competencies  exclusive  to  the  experts: 
Integrity,  Intellectual  Giftedness  and  Curiosity,  Passion  for  Teaching,  and  Devotion  to  the 
Field.  Integrity  describes  a  willingness  to  appraise  one’s  strengths  and  weaknesses 
honestly  and  admit  when  one  is  wrong.  Integrity  emerged  as  the  second  most  important 
competency  on  the  expert  list,  however,  the  novices  rarely  discussed  this  quality.  Since 
the  experts  have  been  in  a  position  to  observe  and  train  other  surgeons  for  many  years, 
they  may  have  had  more  opportunity  to  witness  instances  of  dishonest  behavior  than  did 
the  residents  in  their  limited  experiences.  Residents,  by  contrast,  may  be  less  able  to 
recognize  unethical  behavior  in  a  more  senior  surgeon  and  they  would  not  be  in  a 
position  to  reprimand  another  surgeon  or  terminate  his  or  her  employment.  It  is  likely 
that  novices  will  gain  a  greater  appreciation  for  the  importance  of  integrity  as  their 
training  continues. 

Only  the  experts  noted  the  relationship  between  exceptional  surgical  performance 
and  Intellectual  Giftedness  and  Curiosity.  In  other  words,  experts  believe  that 
exceptional  surgeons  grasp  new  concepts  very  quickly  and  display  an  insatiable 
inquisitiveness  about  the  field.  By  contrast,  most  residents  are  still  struggling  to  learn 
their  specialty.  Therefore,  it  is  possible  that  the  novices  did  not  mention  qualities  related 
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to  intellectual  giftedness  because  they  may  perceive  themselves  at  an  intellectual 
disadvantage  compared  to  more  seasoned  surgeons. 

Furthermore,  only  the  experts  believed  Devotion  to  the  Field  was  a  characteristic 
important  enough  to  be  included  among  their  competencies.  It  is  quite  possible  that  the 
experts  actually  attained  their  levels  of  distinction  within  the  profession  because  they 
possessed  such  strong  devotion.  On  the  other  hand,  the  novices  who  were  interviewed 
were  not  selected  on  the  basis  of  their  achievements.  In  essence,  they  represented  a 
typical  sample  of  surgical  residents.  Therefore,  the  novices  may  not  necessarily  have 
viewed  devotion  to  surgery  as  a  necessary  component  of  a  successful  surgeon. 

Likewise,  it  is  also  not  surprising  that  the  residents  failed  to  mention  the 
relationship  between  surgical  excellence  and  Passion  for  Teaching.  Because  the  novices 
in  this  sample  were  in  the  early  to  middle  years  of  their  residencies,  most  had  not  reached 
the  stage  of  instructing  other  surgeons.  It  is  also  possible  that  many  residents  do  not 
intend  to  remain  in  an  academic  environment  after  they  complete  their  training.  If  that  is 
the  case,  they  are  unlikely  to  view  a  passionate  attitude  toward  education  as  essential  to 
surgical  excellence.  The  majority  of  the  experts  in  this  study  were  academic  surgeons, 
and  more  specifically,  chairs  of  surgery  departments.  As  such,  many  of  them  mentioned 
that  a  love  for  teaching  was  an  important  competency  for  exceptional  surgeons,  although 
it  was  the  lowest-ranking  competency  on  the  expert  list  on  the  expert’s  list. 

Just  as  the  experts  identified  unique  competencies,  the  novices  also  offered  two 
distinct  competencies:  Ability  to  Communicate  in  Nontechnical  Terminology  and 
Maintaining  Composure.  The  Ability  to  Communicate  in  Nontechnical  Terminology  was 
the  highest  ranking  competency  among  the  novices,  but  was  not  mentioned  by  the 
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experts.  It  is  possible  that  novices  emphasized  this  competency  because  they  are  still 
learning  to  put  complicated  concepts  into  language  that  laypeople  can  understand. 
Residents  may  be  able  to  converse  easily  with  other  medical  personnel,  but  they  may  not 
have  enough  experience  with  patients  to  adequately  express  medical  terminology  clearly 
and  succinctly  to  non-experts.  If  this  is  true,  residents  would  be  impressed  by  more 
senior  surgeons  who  can  readily  shift  their  communication  style  to  match  the  needs  of 
their  audience. 

It  is  also  not  surprising  that  residents  viewed  the  ability  to  remain  clam  in  critical 
situations  as  an  essential  characteristic  of  exceptional  surgeons.  The  majority  of  residents 
admitted  that  they  still  become  anxious  during  critical  procedures  or  when  complications 
arise.  They  hold  surgeons  who  are  able  to  maintain  composure  in  high  regard.  Although 
experts  surely  believe  that  is  important  to  remain  calm  in  the  OR,  they  made  very  few 
comments  related  to  this  competency.  Perhaps  it  has  become  second  nature  to  expert 
surgeons  to  maintain  their  composure  in  stressful  situations  having  dealt  with  many  such 
experiences  in  their  careers.  It  is  possible  that  the  residents  will  eventually  regard 
Maintaining  Composure  and  the  Ability  to  Communicate  in  Nontechnical  Terminology 
as  less  essential  to  surgical  excellence  as  they  advance  in  the  careers. 

Similarities  between  Expert  and  Novice  Characteristics  of  a  Poor  Surgeon 

Experts  and  novices  agreed  that  two  attributes  characterize  a  poor  surgeon: 

Apathy  toward  Patient  Care  and  Aggressive  Behavior.  The  majority  of  interviewees 
contended  that  poor  surgeons  often  fail  to  make  patients  their  top  priority.  Both  groups 
commented  that  this  type  of  surgeon  would  rather  shift  patient  schedules  around  to 
accommodate  their  personal  interests  than  adapt  to  their  patients’  needs.  It  is  not 
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surprising  that  experts  and  novices  alike  believed  that  Apathy  toward  Patient  Care  should 
be  on  the  list  of  characteristics  of  defining  a  poor  surgeon  because  both  groups  clearly 
articulated  the  opposite  attitude  toward  patients  in  their  descriptions  of  an  exceptional 
surgeon. 

Both  participant  groups  also  believe  that  poor  surgeons  exhibit  aggressive 
behavior.  Experts  and  novices  alike  commented  that  poor  surgeons  tend  to  yell,  swear, 
and  even  throw  instruments  to  express  their  frustration.  The  majority  of  participants 
could  recall  examples  of  surgeons  they  knew  who  fit  this  mold.  Further,  most 
participants  described  stories  of  surgeons  who  either  acted  aggressively  or  bullied  others 
in  the  OR.  Given  the  frequency  with  which  experts  discussed  belligerent  and  bullying 
surgeons,  it  is  not  surprising  that  both  experts  and  novices  described  aggressiveness  as  a 
characteristic  of  poor  surgeons. 

Differences  between  Expert  and  Novice  Characteristics  of  a  Poor  Surgeon 

One  major  difference  that  distinguished  how  experts  and  novices  defined  a  poor 
surgeon  was  that  the  former  group  commented  on  a  lack  of  honesty.  Dishonesty  can 
manifest  in  several  ways,  including  falsifying  records  and  data,  submitting  the  same 
publications  to  different  journals  or  conferences,  and  performing  surgeries  that  are 
ethically  questionable.  In  most  situations,  residents  are  not  in  a  position  to  know  whether 
other  surgeons  falsify  information  or  submit  dual  publications.  These  activities  could  be 
easily  shielded  from  novice  surgeons  who  are  in  a  subordinate  position  to  senior 
surgeons.  Moreover,  residents  may  not  know  the  qualifications  of  other  surgeons. 
Therefore,  residents  may  assume  that  a  more  senior  surgeon  is  qualified  to  perform  a 
particular  procedure  when  in  reality  he  or  she  is  not.  In  essence,  the  naivete  of  novice 
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surgeons  may  serve  to  obscure  some  of  the  dishonest  actions  of  another  surgeon, 
particularly  when  that  surgeon  is  their  superior.  Experts  also  indicated  that  poor  surgeons 
tend  to  display  a  lack  of  preparation  and  focus  in  the  OR.  According  to  several  expert 
remarks,  many  poor  surgeons  can  be  characterized  as  self-doubting  and  hesitant.  Since 
the  majority  of  surgical  residents  may  also  fit  this  description,  they  may  not  recognize 
these  qualities  as  being  characteristic  of  poor  surgeons. 

Unlike  the  experts,  novices  indicated  that  poor  surgeons  tend  to  demonstrate  a 
lack  of  concern  for  the  feelings  of  patients,  staff  members,  and  other  healthcare 
professionals.  A  possible  explanation  for  why  only  the  novices  commented  on  this 
characteristic  is  that  residents  may  be  part  of  a  new  generation  of  surgeons  who  place 
more  emphasis  on  protecting  the  feelings  of  those  individuals  with  whom  they  interact. 

In  fact,  in  a  recent  Bulletin  of  the  American  College  of  Surgeons,  Chao  and  Wallack 
(2004)  urged  educators  to  “teach  our  residents  to  embrace  the  definition  of  a  surgeon  as  a 
skilled,  compassionate  healer”  (pg.  15).  Given  the  growing  number  of  recent  articles  and 
books  devoted  to  producing  compassionate  young  physicians  (e.g.,  Carmel  &  Glick, 

1996;  Crawshaw,  2002;  Kim,  Kaplowitz,  &  Johnston,  2004),  there  is  good  reason  to 
believe  that  current  residents  may  be  concerned  than  older,  more  experienced  surgeons 
with  developing  communication,  listening,  and  teamwork  skills. 

Similarities  between  Expert  and  Novice  Views  of  an  Exceptional  Surgeon  in  the  Future 

Experts  and  novices  agreed  on  two  themes  that  will  be  important  for  an 
exceptional  surgeon  in  the  future.  The  first  theme  is  that  future  surgeons  must  be  willing 
to  adopt  new  technologies  and  skill  sets  if  they  wish  to  achieve  exceptional  status.  Both 
groups  stated  the  surgeons  of  the  future  will  need  to  stay  abreast  of  the  latest  technology, 
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whether  that  includes  taking  continuing  education  courses  throughout  their  career  or 
practicing  their  skills  in  training  laboratories  that  are  equipped  with  simulators. 

Moreover,  these  surgeons  must  be  able  to  discern  which  technologies  are  most  beneficial 
to  their  patients  and  which  may  be  passing  fads. 

Experts  and  novices  also  agreed  that  the  primary  qualities  of  an  exceptional 
surgeon  will  be  no  different  in  the  future  than  they  are  today.  Both  groups  commented 
that  although  advances  in  technology  will  continue  to  transform  the  field  of  medicine,  the 
core  attributes  of  an  outstanding  surgeon  will  not  change.  Therefore,  the  list  of 
competencies  produced  from  the  present  study  will  probably  be  apropos  to  future 
generations  of  outstanding  surgeons. 

Differences  between  Expert  and  Novice  Views  of  an  Exceptional  Surgeon  in  the  Future 

The  experts  mentioned  four  unique  themes  that  will  be  important  for  an 
exceptional  surgeon  in  the  future  and  the  novices  discussed  two.  The  first  theme  that  was 
exclusive  to  the  experts  concerned  the  impact  of  the  80-hour  work  week.  It  is  surprising 
that  the  novices  did  not  discuss  the  80-hour  work  week  restriction  because  it  affects  them 
directly.  However,  it  is  conceivable  that  many  residents  welcome  the  restriction  in  hours 
since  it  affords  them  more  personal  time.  Because  many  current  residents  have  not  been 
permitted  to  work  more  than  80  hours  per  week,  they  have  no  basis  for  comparison.  By 
contrast,  the  experts  are  from  a  generation  in  which  there  was  no  restriction  on  the 
number  of  hours  they  could  work;  thus,  they  are  able  to  compare  the  new  type  of  surgeon 
with  those  from  previous  generations.  As  stated  earlier,  the  majority  of  experts  are 
unhappy  with  this  restriction  and  are  extremely  concerned  that  future  surgeons  will  be 
less  prepared  and  motivated  than  those  in  the  past. 
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Also,  only  the  experts  mentioned  that  surgeons  in  the  future  are  likely  to  become 
more  specialized.  They  believe  there  will  be  fewer  general  surgeons  as  increasing 
numbers  of  graduates  choose  to  reduce  their  workload  by  concentrating  on  narrower 
surgical  subspecialties.  The  residents  did  not  predict  this  trend  toward  greater 
specialization.  It  is  likely  they  did  not  comment  on  this  issue  because  they  are  too  new  to 
the  field  to  recognize  this  shift  toward  greater  specialization.  In  sum,  residents  may 
simply  lack  the  requisite  experience  to  make  educated  predictions  about  future  trends  in 
surgery. 

Moving  From  Novice  to  Expert 

As  noted  earlier,  several  differences  were  found  between  the  expert  and  novice 
competency  models.  An  important  follow-up  question  is  whether  these  differences 
reflect  a  new  way  of  conceptualizing  surgical  excellence  among  younger  surgeons.  One 
way  to  answer  this  question  would  be  to  compare  views  of  today’s  residents  with  those  of 
other  eras.  Since  that  is  not  possible,  a  second  approach  is  to  examine  views  of  residents 
from  other  eras  based  on  “descriptions”  in  the  literature. 

In  1979,  medical  sociologist,  Charles  Bosk  published  Forgive  and  Remember: 
Managing  Medical  Failure,  a  seminal  book  about  surgical  education.  Bosk  performed  an 
ethnographic  study  in  a  prestigious  American  teaching  hospital.  He  was  interested  in 
uncovering  how  surgeons  perceive,  classify,  and  account  for  medical  errors.  Moreover, 
he  described  how  surgical  faculty  dealt  with  residents’  errors  and  how  residents  modified 
their  behavior  to  avoid  the  wrath  of  attending  surgeons. 

Bosk’s  study  resulted  in  a  categorization  of  four  types  of  medical  errors: 
technical,  judgmental,  normative,  and  quasi-normative.  A  technical  error  reflects  one’s 
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level  of  skill.  Bosk  observed  that  as  long  as  a  surgeon  learns  from  his  or  her  technical 
errors,  they  are  forgivable  within  the  surgical  community.  A  judgmental  error  occurs 
when  a  surgeon  chooses  an  inappropriate  treatment  strategy.  For  instance,  a  surgeon  may 
fail  to  create  a  clear  strategy  for  a  patient  with  a  chronic  condition.  Bosk  found  that  these 
errors  are  also  forgivable  if  a  surgeon  learns  from  them  and  perhaps  undergoes  further 
training.  Normative  errors,  on  the  other  hand,  are  violations  in  the  etiquette  and  norms 
that  come  to  be  expected  between  attending  surgeons  and  residents.  If  a  resident  fails  to 
adhere  to  his  or  her  role,  perhaps  by  not  keeping  the  attending  surgeon  fully  informed  of 
a  patient’s  status,  he  or  she  is  committing  a  normative  error.  Further,  residents  who  treat 
nurses,  staff  members,  or  patients  rudely  are  also  culpable  of  normative  errors.  Bosk 
found  that  these  errors  were  difficult  to  overlook  because  they  are  considered  to  be  a 
direct  reflection  of  a  surgeon’s  integrity  and  ability  to  handle  responsibility.  Finally, 
quasi-normative  errors  occur  when  residents  do  not  follow  the  protocol  of  a  particular 
attending  surgeon.  In  essence,  residents  who  commit  these  errors  appear  insubordinate  to 
authority  figures.  When  residents  blatantly  disregard  the  expectations  of  their  attending 
surgeon,  they  are  committing  a  quasi-normative  error. 

Bosk  observed  that  technical  or  judgmental  errors  made  by  residents  were 
tolerated  and  even  expected.  They  were  forgivable  as  long  as  they  were  not  repeated.  By 
contrast,  normative  and  quasi-normative  errors  could  actually  have  a  deleterious  effect  on 
a  surgeon’s  career.  An  occasional  technical  or  judgmental  error  that  occurred  on  was  not 
likely  to  have  a  lasting  negative  impact  as  long  as  the  resident  did  not  attempt  to  cover  up 
the  error.  Bosk  found  that  surgeons  could  be  transferred  or  even  fired  if  they  were  not 
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honest  about  the  errors  they  committed  or  if  they  did  not  interact  appropriately  and 
respectfully  with  staff  and  patients. 

Two  of  the  four  types  of  errors  that  Bosk  identified  were  related  to  the 
competencies  in  the  present  model.  In  particular,  normative  and  quasi-normative  errors 
are  directly  related  to  the  model.  Surgeons  whose  integrity  is  questioned  are  often 
viewed  as  unreliable  and  untrustworthy.  In  his  study,  Bosk  noted  that  several  surgeons 
were  placed  out  of  prestigious  hospitals  for  demonstrating  “a  lack  of  integrity  and  a 
desire  to  take  the  easy  way  out”  (p.  162).  In  addition,  Bosk  revealed  that  surgeons  would 
also  be  removed  from  esteemed  programs  if  their  interactions  with  patients,  staff 
members,  and  authority  figures  could  not  be  trusted.  Therefore,  in  Bosk’s  study, 
surgeons  were  expected  to  demonstrate  the  competencies,  Integrity,  Humility,  and 
Compassion  identified  in  the  present  model. 

The  views  of  experts  in  the  present  study  support  Bosk’s  findings  in  that  they  both 
highlight  the  notion  that  character  flaws  are  more  likely  to  prevent  a  surgeon  from 
achieving  excellence  than  a  lack  of  technical  ability.  Bosk’s  findings  suggest  that  most 
surgical  faculty  expect  inexperienced  surgeons  to  commit  technical  and  judgmental 
errors.  In  fact,  none  of  the  experts  in  the  present  study  mentioned  that  they  had 
reprimanded  or  fired  surgeons  for  making  technical  errors.  However,  several  experts 
commented  that  they  had  fired  surgeons  due  to  their  dishonest  behavior,  such  as 
falsifying  data  in  a  research  study. 

Bosk  also  observed  how  the  values  of  the  attending  surgeons  influenced  the 
behavior  of  the  residents  in  his  study.  The  attending  surgeons  expected  residents  to 
respect  the  chain  of  command  that  exists  within  every  surgical  service.  Consequently,  a 
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resident  must  be  willing  to  humble  himself  and  “acknowledge  his  subordination  to  the 
group  and  its  standards”  (p.  180).  Residents  who  failed  to  adjust  their  behavior  and 
accept  their  position  in  the  service  could  be  asked  to  leave.  The  values  of  the  attending 
surgeons  observed  by  Bosk  are  similar  to  those  espoused  by  the  experts  in  this  study. 
They  both  believed  it  was  important  for  surgeons  to  display  humility  and  respect  toward 
other  healthcare  professionals.  According  to  many  of  the  experts,  poor  surgeons  not  only 
lack  humility,  but  they  often  disparage  other  surgeons  and  staff  members. 

One  of  the  main  findings  from  Bosk’s  study  was  that  surgeons  of  that  generation 
viewed  technical  errors  as  “easily  correctable  shortcomings”  but  errors  of  morality  and 
character  were  punishable  offenses  (p.  175).  It  is  conceivable  that  today’s  surgical 
residents  espouse  a  somewhat  different  view  of  surgical  excellence  than  their 
predecessors.  Recall  that  in  the  present  study  the  novice  competency  model  differed 
from  the  expert  model  in  that  it  placed  greater  emphasis  on  effective  communication  with 
patients,  a  strong  bedside  manner,  and  sociability.  Other  than  these  differences,  there 
was  a  good  deal  of  overlap  between  the  two  models.  It  is  possible  that  younger  surgeons 
simply  value  interpersonal  skills  more  than  surgeons  from  previous  generations.  A 
second  explanation  is  that  novice  surgeons  are  still  idealistic  in  their  thinking  and 
therefore,  their  view  of  the  qualities  that  comprise  an  outstanding  surgeon  do  not  yet 
coincide  with  those  of  more  experienced  surgeons.  Thus,  an  interesting  follow-up 
question  is  whether  surgical  residents  will  consistently  regard  communication  and 
sociability  as  important  competencies  throughout  their  career. 
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Biography  of  an  Outstanding  Surgeon 

One  of  the  most  renowned  surgeons  of  the  past  century,  Denton  Cooley,  was  a 
pioneer  in  cardiovascular  surgery.  In  1968,  Cooley  performed  the  first  successful  human 
heart  transplant  in  the  United  States.  The  following  year  he  became  the  first  heart 
surgeon  to  implant  an  artificial  heart  into  a  man.  Cooley’s  biography  reveals  a  detailed 
account  of  his  personal  and  professional  accomplishments  (Minetree,  1973).  Since  there 
are  few  biographies  that  focus  on  outstanding  surgeons,  this  particular  one  provides  a  rare 
opportunity  to  examine  the  characteristics  of  a  truly  exceptional  surgeon.  According  to 
Minetree,  Cooley  shares  many  of  the  attributes  that  were  found  in  the  present  competency 
model. 

One  of  Cooley’s  most  salient  qualities  was  his  absolute  commitment  to  the 
profession  and  his  patients.  In  fact,  Minetree  (1973)  noted  that  Cooley  was  “a  man  who 
spent  more  time  inside  human  hearts  than  his  own  home”  (p.  22).  Cooley  even  insisted 
that  his  family  vacations  be  spent  in  locations  near  hospitals  that  were  equipped  for  heart 
surgery.  Minetree  noted  that  Cooley’s  personal  and  familial  obligations  nearly  always 
came  second  to  his  work.  In  fact,  Cooley’s  dedication  to  his  patients  prevented  him  from 
spending  a  lot  of  time  with  his  five  daughters.  This  quality  is  akin  to  the  Dedication  to 
Patient  Care  and  Devotion  to  Field  competencies  within  the  present  model.  Minetree  also 
provided  many  examples  of  Cooley’s  conscientious  nature.  In  fact,  Cooley  referred  to 
himself  as  a  “work  addict”  who  became  depressed  during  slow  periods.  He  would  use 
gaps  in  his  work  schedule  to  perfect  complicated  operations.  This  compulsive  tendency 
reflects  the  Tireless  Work  Ethic  competency  identified  in  the  present  model. 
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An  additional  competency  that  Cooley  appeared  to  reflect  was  Preparedness. 
Cooley  was  a  surgeon  who  had  a  tremendous  capacity  for  pressure.  Despite  the  fact  that 
he  was  participating  in  ground-breaking  and  life-threatening  procedures,  he  was 
consistently  described  as  not  only  having  the  ability  to  remain  calm,  but  he  was  able  to 
rally  his  surgical  team  under  periods  of  high  pressure.  Cooley  also  displayed 
Compassion.  In  an  interview  with  Minetree,  Cooley  noted  that  he  tried  to  temper  his 
comments  toward  young  surgeons  when  they  made  mistakes  in  the  OR.  He  stressed  the 
importance  of  handling  these  situations  with  patience  and  understanding.  Finally,  Cooley 
was  described  throughout  the  biography  as  a  highly  intelligent  man.  He  also  possessed 
an  insatiable  curiosity  about  cardiac  surgery  and  devotion  toward  developing  new  and 
improved  heart  procedures.  Therefore,  Cooley  also  mirrored  the  competency,  Intellectual 
Giftedness  and  Curiosity. 

Cooley  also  possessed  several  characteristics  that  were  not  reflected  in  the  present 
competency  model.  For  instance,  Minetree  described  Cooley  as  having  “a  sense  of 
competitiveness  that  bordered  on  fanatic”  (p.  23).  Although  it  is  likely  that  a  successful 
person  in  any  profession  displays  some  degree  of  competitiveness,  a  competitive  nature 
did  not  emerge  within  the  present  model.  In  addition,  Minetree  described  Cooley  as  a 
“technical  wizard.”  Although  technical  ability  is  clearly  necessary  for  surgery,  it  did  not 
reach  criteria  for  inclusion  in  the  final  competency  model. 

Comparison  of  the  Expert  Competency  List  and  the  ACGME  Competencies 

The  ACGME  established  a  list  of  six  competencies  that  residents  are  expected  to 
master  throughout  their  medical  training  (Dunnington  &  Williams,  2003).  It  was 
predicted  that  there  would  be  some  overlap  between  the  ACGME  competencies  and  the 
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expert  list  of  competencies,  but  that  additional  knowledge,  skills,  and  abilities  would 
distinguish  the  two  lists.  In  other  words,  it  was  anticipated  that  experts  would  identify 
additional  KSAOs  unique  to  exceptional  surgeons. 

Similarities  between  the  Expert  List  of  Competencies  and  ACGME  Competencies 
Interestingly,  none  of  the  experts  mentioned  the  ACGME  competencies  during 
their  interviews.  One  possible  reason  that  the  experts  did  not  mention  this  topic  was  that 
the  interviewer  did  not  use  the  word  “competency”  in  the  actual  interview  questions. 
Rather,  experts  were  asked  to  think  of  the  finest  surgeons  they  knew  and  explain  the 
rationale  behind  their  choices.  It  is  more  common  and  effective  to  use  this  phrasing  in  a 
competency  interview  rather  than  to  simply  ask  participants  to  list  the  qualities  that  they 
would  deem  to  be  competencies  in  their  profession  (Lucia  &  Lepsinger,  1999).  In 
addition,  this  format  personalizes  the  interview  process  for  the  participants  and  thus 
generates  more  discussion. 

One  of  the  most  noticeable  similarities  between  the  expert  and  ACGME  lists  is 
that  they  both  emphasize  patient  treatment.  The  first  ACGME  competency,  Patient  Care, 
refers  to  patient  treatment  that  is  “compassionate,  appropriate,  and  effective  for  the 
treatment  of  health  problems  and  the  promotion  of  health.”  The  expert  competencies, 
Compassion  and  Dedication  to  Patient  Care,  are  clearly  linked  to  the  ACGME 
competency  of  Patient  Care.  A  second  ACGME  competency,  Interpersonal  and 
Communication  Skills,  is  also  related  to  Compassion  and  Dedication  to  Patient  Care,  but 
perhaps  less  directly  than  the  aforementioned  ACGME  competency.  The  ACGME 
defined  Interpersonal  and  Communication  Skills  as  those  “skills  that  result  in  effective 
information  exchange  and  collaboration  with  patients,  their  families,  and  other  healthcare 
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professionals.”  It  involves  both  caring  and  effective  communication  with  others. 
Although  the  experts  did  not  offer  a  competency  that  focused  exclusively  on 
communication  skills,  aspects  of  communication  components  were  subsumed  within 
Compassion  and  Dedication  to  Patient  Care. 

Another  ACGME  competency  related  to  the  expert  competencies  was  Medical 
Knowledge.  The  ACGME  defined  this  competency  as  “medical  knowledge  about 
established  and  evolved  biomedical,  clinical,  and  cognate  sciences  and  the  application  of 
this  knowledge  to  patient  care.”  Although  the  term,  medical  knowledge,  was  not 
explicitly  used  by  the  experts,  it  was  subsumed  in  several  of  their  competencies.  For 
instance,  medical  knowledge  involves  the  use  of  scientific  and  clinical  information  in 
operative  decision  making  and  patient  management.  It  also  concerns  the  ability  to 
demonstrate  basic  and  clinical  knowledge,  as  well  as  critically  evaluate  scientific 
information.  Therefore,  this  definition  is  somewhat  related  to  the  expert  competencies, 
Rapid  Decision  Making  and  Intellectual  Giftedness  and  Curiosity. 

Professionalism  is  defined  by  the  ACGME  as  “commitment  to  carrying  out 
professional  responsibilities,  adherence  to  ethical  principles,  and  sensitivity  to  a  diverse 
population.”  Thus,  Professionalism  is  directly  related  to  the  expert  competency, 

Integrity.  Although  the  ACGME  competencies  and  those  of  the  experts  are  referred  to  by 
different  names,  their  definitions  are  interrelated.  Both  competencies  address  the  ethical 
conduct  of  physicians  as  it  relates  to  their  patients  and  fellow  healthcare  professionals. 
Furthermore,  elements  of  Humility  may  be  subsumed  within  Professionalism.  Humility 
refers  to  treating  patients  and  healthcare  professionals  with  respect  regardless  of  their 
rank.  Likewise,  Professionalism  involves  treating  patients  and  other  healthcare  workers 
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with  respect  and  sensitivity,  particularly  in  regard  to  their  age,  gender,  and  cultural 
background. 

The  ACGME  competency,  Practice-Based  Learning  and  Improvement,  is  defined 
as  “the  investigation  and  evaluation  of  one’s  own  patient  care,  appraisal  and  assimilation 
of  scientific  evidence,  and  improvement  of  patient  care.”  According  to  this  ACGME 
competency,  physicians  must  be  able  to  integrate  scientific  evidence  when  diagnosing 
patients  and  developing  treatment  plans.  This  notion  is  related  to  the  expert  competency, 
Rapid  Decision  making,  which  involves  the  ability  to  make  the  best  decisions  for  patients 
based  on  available  data.  Furthermore,  “the  evaluation  of  one’s  own  patient  care”  is 
related  to  the  expert  competency,  Integrity,  which  involves  taking  an  honest  look  at  one’s 
strengths  and  weaknesses.  Finally,  Practice-Based  Learning  and  Improvement  involves 
the  effective  use  of  information  technology  to  manage  information.  Although  the 
competencies  did  not  mention  the  importance  of  using  technology,  the  experts  discussed 
this  topic  as  a  theme  impacting  future  surgeons.  Specifically,  Practice-Based  Learning 
and  Improvement  is  directly  related  to  the  expert  theme,  Willingness  to  Adopt  New 
Technologies  and  Skill  Sets. 

Differences  between  the  Expert  List  of  Competencies  and  ACGME  Competencies 
Only  one  ACGME  competency,  System-Based  Learning,  was  unrelated  to  the 
expert  competencies.  Recall  that  the  ACGME  defines  System-Based  Learning  as 
“system-based  practice  manifested  by  actions  that  demonstrate  an  awareness  of  and 
response  to  the  larger  context  and  system  of  health  care  and  effectively  call  on  system 
resources  to  provide  optimal  care.”  This  competency  may  be  demonstrated  by  an 
understanding  of  the  role  played  by  different  specialties  and  other  health  care 
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professionals  in  overall  patient  management.  It  could  also  reflect  the  ability  to  access  and 
mobilize  outside  resources.  Although  the  experts  probably  deemed  an  awareness  of  and 
response  to  the  health  care  system  as  an  important  quality  for  surgeons,  they  did  not 
appear  to  think  it  would  be  sufficient  to  distinguish  between  experts  and  average 
surgeons.  An  understanding  of  the  system  in  which  surgery  is  practiced  is  important 
knowledge  that  surgeons  should  attain  during  their  career,  but  possessing  that  knowledge 
most  likely  does  not  ensure  surgical  excellence. 

Several  expert  competencies  emerged  that  were  not  among  the  ACGME 
competencies.  Specifically,  Tireless  Work  Ethic,  Devotion  to  Field,  and  Passion  for 
Teaching  were  not  included  in  the  ACGME  list.  Tireless  Work  Ethic  and  Devotion  to 
Field  reflect  the  vigorous  efforts  of  exceptional  surgeons  to  excel  in  their  profession  and 
propel  the  field  forward.  Since  a  purpose  of  the  ACGME  competencies  is  to  guide  the 
educational  and  training  objectives  used  across  academic  institutions,  it  would  be 
difficult,  and  possibly  inappropriate,  to  assess  physicians’  level  of  devotion  or 
commitment  to  the  field  unless,  of  course,  their  lack  of  commitment  compromises  their 
ability  to  treat  patients.  Similarly,  the  ACGME  cannot  mandate  that  physicians  be 
evaluated  on  their  love  of  teaching,  especially  since  many  physicians  in  training  do  not 
intend  to  remain  in  an  academic  setting  or  teaching  hospital  upon  graduation. 

Nonetheless,  the  experts  believed  that  a  passionate  attitude  toward  education  was  a 
characteristic  that  distinguishes  between  outstanding  and  typical  surgeons. 
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Caveats  of  the  Present  Study 

The  present  research  is  unique  because  it  is  the  first  of  its  kind  to  distill  interview 
data  from  twenty-six  expert  surgeons  into  a  detailed  model  that  describes  the  knowledge, 
skills,  abilities,  and  other  characteristics  (KSAOs)  associated  with  surgical  excellence. 
However,  given  the  nature  of  the  competency  modeling  approach,  there  are  several 
concerns  regarding  the  external  validity  of  the  results  that  should  be  noted.  Most  of  these 
concerns  are  tied  to  the  characteristics  of  the  sample.  For  instance,  the  majority  of 
experts  who  participated  were  from  academic  institutions.  Specifically,  22  surgeons  were 
drawn  from  academic  institutions  but  only  4  were  drawn  from  private  practice.  Given 
that  some  of  the  criteria  for  inclusion  in  the  study  were  having  multiple  literature 
citations,  editing  textbooks,  and  holding  offices  or  titles  in  professional  societies,  it 
should  not  be  surprising  that  the  majority  of  participants  were  drawn  from  teaching 
hospitals  and  academic  institutions.  Consequently,  comments  from  surgeons  working 
strictly  in  private  practice  were  limited.  However,  a  comparison  of  the  information 
provided  by  experts  from  these  two  settings  yielded  no  appreciable  differences.  In  fact, 
two  of  the  experts  from  private  practice  commented  that  outstanding  surgeons  tend  to 
have  a  great  love  for  teaching.  Therefore,  the  limited  representation  from  nonacademic 
surgeons  probably  had  minimal  impact  on  the  final  competency  model. 

Another  concern  regarding  the  sample  was  that  it  contained  only  one  female 
expert.  Given  that  most  of  the  participants  were  at  least  50  years  of  age,  they  finished 
their  residencies  in  a  time  when  males  represented  the  vast  majority  of  surgeons.  Thus,  it 
is  not  surprising  that  the  expert  sample  was  predominantly  male.  The  low  female 
representation  probably  did  not  have  a  significant  impact  on  the  results  because  the 
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competencies  identified  were  sex  indifferent.  In  other  words,  none  of  the  competencies 
could  be  labeled  as  strictly  male  or  female  issues.  In  fact,  the  comments  from  the  female 
participant  in  the  sample  were  indistinguishable  from  the  comments  made  by  the  male 
participants.  More  specifically,  her  comments  reflected  seven  of  the  ten  final 
competencies.  Therefore,  there  is  little  reason  to  believe  that  additional  female  experts  in 
the  sample  would  have  had  any  appreciable  impact  on  the  results. 

Unlike  the  experts,  however,  fifty  percent  of  the  participants  in  the  novice  sample 
were  women.  The  different  proportions  of  males  and  females  in  the  two  samples  reflect 
actual  demographic  shifts  in  surgery.  In  fact,  the  Association  of  American  Medical 
Colleges  reported  that  as  of  1999,  the  number  of  women  in  general  surgery  had  increased 
to  21%  (Baumgartner,  Tseng,  &  DeAngelis,  2001).  The  greater  number  of  females  in  the 
novice  group  allowed  for  a  direct  comparison  between  male  and  female  comments.  Once 
again,  there  was  nothing  unique  about  the  comments  from  females  and  males.  Both  male 
and  female  comments  reflected  a  majority  of  the  competencies  that  were  included  in  the 
final  model.  Interestingly,  males  and  females  placed  equal  emphasis  on  the  importance 
of  effective  communication,  empathy  toward  patients,  and  an  excellent  bedside  manner. 

The  results  from  the  present  study  provide  the  foundation  of  a  model  that  could  be 
used  for  selecting  and  training  excellent  surgeons.  However,  the  model  still  needs  to  be 
validated.  It  is  important  to  emphasize  that  the  approach  taken  in  the  present  study 
represents  the  initial  phase  in  this  process.  Since  a  job  analysis  or  competency  model  for 
surgeons  does  not  presently  exist,  it  was  necessary  to  create  such  a  model  before  any 
selection,  training,  or  appraisal  methods  could  be  developed.  Thus,  this  study  should 
serve  as  a  platform  for  future  quantitative  projects.  For  example,  it  would  be  important  to 
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determine  whether  each  competency  measures  a  unique  construct  or  whether  there  is 
some  overlap  among  them.  Consequently,  a  questionnaire  based  on  the  competencies 
would  need  to  be  developed  so  that  each  competency  could  be  examined  in  more  detail. 
The  questionnaire  would  then  be  administered  to  a  much  larger  sample  of  surgeons  than 
used  in  the  present  study.  The  results  from  the  questionnaire  would  be  subjected  to  a 
confirmatory  factor  analysis  (CFA)  to  determine  whether  each  competency  (i.e., 
subscale)  measures  a  single  construct.  The  CFA  procedure  is  important  because  it  can 
establish  whether  the  number  of  dimensions  or  competencies  conforms  to  the  number 
found  in  the  model.  Further,  the  CFA  would  show  how  the  ten  competencies  could  be 
condensed  into  fewer  dimensions  if  overlap  exists  among  them. 

In  addition,  the  subscales  could  be  correlated  with  existing  and  well-validated 
measures.  For  example,  to  establish  construct  validity,  the  Rapid  Decision  Making  and 
Preparedness  subscales  might  be  correlated  with  the  California  Critical  Thinking  Skills 
Test  (CCTST;  Facione  &  Facione,  1998)  described  above  since  its  scales  measure  many 
of  the  important  characteristics  and  behaviors  related  to  these  competencies.  For 
instance,  one  scale  on  the  CCTST  focuses  on  inductive  reasoning.  An  important  aspect 
of  inductive  reasoning  concerns  how  individuals  use  their  past  experiences  to  make 
predictions  about  future  events  (Facione,  1990).  Thus,  one  would  expect  this  subscale  of 
the  CCTST  to  be  positively  correlated  with  the  competencies,  Rapid  Decision  Making 
and  Preparedness,  identified  in  the  present  study. 

To  determine  construct  validity,  it  is  also  necessary  to  establish  discriminant 
validity  (Cascio,  1998),  that  is,  the  absence  of  correlations  among  measures  of  unrelated 
theoretical  constructs  (Campbell  &  Fiske,  1959).  Therefore,  the  subscales  derived  from 
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these  competencies  should  also  be  assessed  with  established  tests  of  unrelated  constructs. 
For  example,  Rapid  Decision  Making  and  Preparedness  might  be  evaluated  with  a 
measure  of  emotional  intelligence  because  critical  thinking  is  considered  to  be  unrelated 
to  emotional  intelligence. 

An  additional  use  of  such  a  questionnaire  would  be  to  discriminate  between 
outstanding  and  typical  surgeons.  Since  competencies  are  those  KSAOs  that  have  the 
most  direct  impact  on  success  for  a  surgeon,  the  model  should  distinguish  among 
surgeons.  In  many  organizations,  competencies  are  often  correlated  with  performance 
measures  such  as  productivity,  increases  in  sales  or  profit,  and  performance  ratings 
(Lucia  &  Lepsinger,  1999).  In  the  case  of  a  surgeon,  performance  measures  are  a  more 
complicated  concern  because  measurable  indicators  of  performance  are  less  common. 
Nonetheless,  the  competencies  could  be  correlated  with  patient  satisfaction  scores, 
complication  rates,  or  scores  received  from  standardized  patient  encounters. 

Implications  for  Training  and  Selection 

Once  competencies  are  identified,  they  are  often  used  to  guide  and  improve 
training  and  selection  procedures  for  organizations.  Likewise,  the  results  of  the  present 
competency  model  could  be  incorporated  into  residency  programs.  The  following  section 
provides  some  examples  of  how  training  and  selection  processes  for  surgical  residency 
programs  could  benefit  from  the  competencies  identified  in  the  present  study. 

Enhancing  the  Training  Process 

Once  competencies  are  identified  and  validated,  the  next  logical  step  is  to 
consider  how  they  could  be  used  to  train  outstanding  surgeons.  In  other  words,  what  type 
of  training  could  be  implemented  to  help  typical  surgeons  become  excellent  surgeons? 
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Interpersonal  Skills  Training 

Based  upon  the  present  competency  model,  it  is  reasonable  to  believe  that 
surgeons  are  more  likely  to  achieve  excellence  if  they  possess  strong  interpersonal  skills. 
Therefore,  it  is  recommended  that  some  form  of  interpersonal  skills  training  be 
incorporated  into  the  curriculum.  One  method  that  is  currently  used  for  interpersonal 
skill  training  and  evaluation  is  "standardized  patient  encounters"  (Ainsworth,  Rogers, 
Markus,  Dorsey,  Blackwell,  &  Petrusa,  1991).  Standardized  patients  are  healthy 
individuals  who  are  trained  to  portray  patients  by  simulating  the  symptoms  of  a  variety  of 
illnesses,  as  well  as  the  history,  personality,  and  response  patterns  of  typical  patients. 
Although  standardized  patient  encounters  are  well  accepted  as  a  method  for  improving 
interpersonal  skills,  their  use  as  a  formal  method  has  yet  to  be  fully  integrated  into  many 
medical  schools  or  surgical  residency  curricula  (Yedidia  et  al.,  2003).  Because  the 
present  results  indicate  that  there  is  a  strong  link  between  strong  interpersonal  skills  and 
surgical  excellence,  it  is  recommended  that  validated  standardized  patient  encounters  be 
incorporated  into  surgical  residency  training. 

In  addition  to  the  use  of  standardized  patients,  medical  schools  and  residency 
programs  should  place  greater  emphasis  on  communication  skills.  On  an  informal  level, 
it  is  important  for  experienced  surgeons  to  use  casual  opportunities  that  arise  during 
clinical  activities  to  model  good  interpersonal  skills.  On  a  more  formal  basis,  classes  or 
workshops  on  communication  skills  could  be  offered.  Further,  it  is  important  to  provide 
regular  assessment  and  feedback  to  ensure  that  these  skills  are  learned  and  maintained. 
One  instrument  that  might  be  used  to  assess  interpersonal  skills  is  the  Doctors’ 
Interpersonal  Skills  Questionnaire  (DISQ;  Greco,  Sweeney,  Broomhall,  &  Beasley, 
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2001).  This  instrument  may  be  completed  by  patients  or  surgeons  who  are  in  a  position 
to  train  and  evaluate  residents.  Two  examples  of  items  on  the  DISQ  include,  “On  this 

visit  I  would  rate  the  doctor’s  ability  to  really  listen  as _ ”  and  “The  respect  shown  to 

be  by  this  doctor  was _ When  responding  to  these  questions,  individuals  are  given 

a  scale  that  ranges  from  “poor”  to  “excellent.” 

Professional  Ethics  Seminars 

Given  the  frequency  with  which  the  experts  discussed  the  importance  of  Integrity, 
it  is  also  suggested  that  surgical  residents  take  a  seminar  in  ethical  training  during  their 
residencies.  By  exposing  residents  to  this  topic,  faculty  members  will  demonstrate  that 
they  take  professional  ethics  very  seriously.  Such  seminars  should  provide  theoretical 
foundations,  as  well  as  practical  skills  required  for  ethical  reasoning  (Hattab,  2004). 
Potential  topics  for  discussion  during  professional  ethics  seminars  could  include  informed 
consent,  end-of-life  care,  and  methods  for  dealing  with  medical  error.  It  may  also  be 
beneficial  to  include  role  play  opportunities  in  these  seminars. 

Enhancing  the  Selection  Process 

Residents  can  acquire  many  important  skills  through  on-the-job  training  and 
formal  coursework.  However,  there  are  some  important  characteristics  that  may  be  less 
amenable  to  training,  such  as  the  character-related  competencies  found  in  the  present 
study  (e.g.,  Compassion,  Humility,  and  Integrity).  When  training  alone  may  be 
inappropriate  or  insufficient  for  certain  competencies,  it  may  be  worth  considering  new 
methods  to  enhance  the  selection  process  used  by  many  surgical  residency  programs.  The 
following  section  contains  several  recommendations  for  improving  the  selection  of 
surgical  residents. 
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Critical  Thinking 

Given  that  Decision  Making  and  Preparedness  emerged  as  important 
competencies,  it  is  recommended  that  residency  programs  also  include  a  measure  of 
critical  thinking  or  decision  making  ability  in  their  selection  process.  As  mentioned 
earlier,  the  CCTST  is  a  standardized  and  validated  multiple-choice  measure  of  critical 
thinking  skill  (Facione  &  Facione,  1998).  Although  it  is  considered  to  be  a  discipline- 
neutral  test,  it  has  been  used  previously  in  medical  research  (Bondy,  Koenigseder,  Ishee, 
&  Williams,  2001).  Just  as  this  measure  would  be  appropriate  for  establishing  evidence 
of  construct  validity,  it  would  also  be  a  useful  selection  device  because  it  measures  many 
of  the  important  characteristics  related  to  critical  thinking  (e.g.,  inductive  and  deductive 
reasoning).  An  alternative  to  the  CCTST  could  be  a  context-specific  decision  making 
test.  If  a  context-specific  measure  is  used  to  assess  critical  thinking,  subject  matter 
experts  should  be  instrumental  in  its  formation. 

Characteristics  of  a  Poor  Surgeon 

The  experts  identified  several  characteristics  of  a  poor  surgeon,  including  Apathy 
toward  Patients,  Dishonesty,  and  a  “clock-watching”  or  work-shift  mentality.  If  the 
selection  process  could  distinguish  applicants  who  possess  these  characteristics, 
residency  programs  would  save  a  great  deal  of  time  and  resources  because,  if  selected, 
these  individuals  might  eventually  be  removed  from  the  program  or  require  additional 
attention  and  training.  One  way  to  identify  candidates  with  a  tendency  toward  apathetic 
or  uncaring  attitudes  is  to  administer  an  emotional  intelligence  test,  such  as  the  Bar-On 
Emotional  Quotient  Inventory  (EQi;  Wagner,  Moseley,  Grant,  Gore,  &  Owens,  2002). 

In  particular,  the  Interpersonal  and  Stress  Management  Scales  of  the  EQi  would  be 
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appropriate  to  use  in  this  context.  The  former  measures  one’s  ability  to  be  empathetic  and 
develop  strong  interpersonal  relationships  while  the  latter  assesses  impulse  control  and 
stress  tolerance.  Questions  from  an  emotional  intelligence  test  could  also  be  adapted  into 
structured  interview  questions. 

Characteristics  Unique  to  a  Competency  Model  for  Surgery 

One  might  expect  that  the  present  competency  model  is  unique  to  surgery.  In 
order  to  make  this  assertion,  it  is  necessary  to  compare  the  present  model  to  models  from 
other  domains.  Lucia  and  Lepsinger  (1999)  created  and  validated  several  models  for 
various  professional  roles,  including  manager,  salesperson,  finance  employee,  attorney, 
and  research  associate  in  a  consulting  firm.  Some  of  the  competencies  listed  for  a 
manager  include  Informing,  Monitoring,  Delegating,  Planning,  Problem  Solving,  and 
Mentoring.  Clearly,  this  competency  model  differs  from  the  surgeon’s  model. 

According  to  Lucia  and  Lepsinger’ s  research,  a  successful  manager  should  disseminate 
relevant  information  to  employees,  review  their  progress,  assign  responsibilities, 
determine  strategic  objectives,  systematically  analyze  problems,  and  facilitate  employees’ 
career  growth.  Unlike  the  model  for  a  surgeon,  there  were  no  competencies  that  related 
to  one’s  character  or  personality.  However,  many  of  the  competencies  for  a  manager 
relate  to  one’s  obligation  toward  subordinates.  While  most  of  the  competencies  for  a 
surgeon  are  not  related  to  their  subordinates  (i.e.,  residents),  they  are  directly  related  to 
patients. 

The  competency  model  for  an  attorney  is  closer  in  scope  to  the  surgeon’s  model. 
Some  of  the  competencies  for  an  attorney  include  Writing  Skills,  Oral  Communication 
Skills,  Analytical  Thinking,  Detail  Handling,  Planning  and  Organizing,  Influencing  and 
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Negotiating,  Competitiveness,  Reflectiveness/Introspection,  and  Endurance.  Models  for 
attorneys  and  surgeons  are  similar  in  that  they  both  emphasize  Analytical  (or  critical) 
Thinking,  Planning  and  Organizing  (or  Preparedness),  Endurance  (or  Tireless  Work 
Ethic)  and  Reflectiveness/Introspection  (Integrity).  In  fact,  Reflectiveness/Introspection 
for  attorneys  refers  to  a  willingness  to  admit  to  mistakes.  However,  the  experts  in  the 
present  study  did  not  comment  on  the  importance  of  written  or  oral  communication  skills, 
nor  did  they  mention  Detail  Handling,  Influencing/  Negotiating,  or  Competitiveness. 

The  present  model  appears  to  be  different  from  models  in  other  domains  for 
several  reasons.  First,  surgeons  are  expected  to  make  rapid,  well-informed  decisions  in 
high  pressure  situations.  In  addition,  the  field  of  medicine  is  one  of  the  few  domains  in 
which  a  significant  part  of  the  job  involves  the  ability  to  relate  well  to  patients  and  their 
family  members.  Consequently,  the  present  model  is  also  unique  in  that  successful 
surgeons  are  expected  to  demonstrate  empathy  and  to  allow  their  relationships  with 
patients  to  take  priority  over  their  personal  needs  and  obligations. 
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CONCLUSIONS 

The  present  study  was  designed  to  identify  the  competencies  associated  with 
excellence  in  surgery.  Twenty-six  expert  surgeons  were  interviewed  and  their  comments 
aggregated  to  establish  a  collective  opinion  of  the  KSAOs  required  of  an  outstanding 
surgeon.  The  interviews  resulted  in  10  competencies  revealing  that  outside  of 
intelligence,  success  in  surgeons  is  largely  contingent  upon  their  interpersonal  and 
communication  skills. 

As  noted  above  in  Dr.  William  Silen’s  keynote  address  at  the  2003  Annual 
Meeting  of  the  Society  of  University  Surgeons,  he  stated  that  the  apprenticeship  model  of 
training  surgeons  is  no  longer  viable  and  that  there  is  almost  no  time  available  for 
practicing,  teaching,  and  thinking.  Not  only  is  the  “see  one,  do  one,  teach  one”  approach 
to  surgical  training  now  considered  obsolete,  but  even  with  the  80-hour  work  week 
restriction  many  residents  find  themselves  drained  by  their  work  schedules.  In  the 
present  study,  it  was  extremely  difficult  to  find  novice  surgeons  to  participate  in  the 
30-minute  interview.  It  took  twice  as  long  to  recruit  half  as  many  novices  as  experts.  In 
fact,  many  novices  agreed  to  participate  but  had  to  withdraw  due  to  their  hectic 
schedules.  Silen’s  point  appears  to  be  true.  Many  residents  seem  overwhelmed  by  their 
workload  and  simply  do  not  have  enough  time  to  sleep,  much  less  participate  in  a 
voluntary  interview. 

The  process  of  executing  the  present  study,  as  well  as  the  results  obtained,  also 
support  the  idea  that  what  is  needed  to  train  the  next  generation  of  surgeons  may  be 
changing.  At  the  outset,  it  was  expected  that  exceptional  surgeons  would  differ  from 
others  in  their  intellectual  abilities  and  technical  skills.  The  results  provide  only  partial 
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support  for  this  view.  On  the  one  hand,  Intellectual  Giftedness,  Preparedness,  and  Rapid 
Decision  Making  are  clearly  related  to  excellence  in  surgery.  On  the  other  hand,  the 
importance  of  technical  skills  was  not  emphasized  in  the  competency  model.  This  does 
not  suggest  that  manual  dexterity  and  procedural  skills  are  unimportant.  Rather,  technical 
skill  may  be  necessary,  but  not  sufficient  to  achieve  surgical  excellence.  Instead,  many 
of  the  qualities  that  set  exceptional  surgeons  apart  from  the  rest  are  related  to  one’s 
character.  The  present  model  demonstrates  that  a  truly  outstanding  surgeon  must  also 
display  compassion,  humility,  and  integrity.  Historically,  these  characteristics  have  not 
received  much  attention  in  surgical  education.  Thus,  the  traditional  Halstedian  training 
paradigm  that  stresses  procedural  skills  may  be  ill-suited  to  produce  the  breadth  of 
qualities  expected  in  exceptional  surgeons. 

Surgical  educators  may  need  to  consider  offering  mandatory  courses  or 
workshops  to  help  residents  recognize  and  develop  the  behaviors  associated  with  these 
competencies.  On  the  other  hand,  the  present  model  revealed  that  there  are  certain 
characteristics  that  educators  will  not  be  able  to  address  through  formal  educational 
methods.  For  example,  surgical  excellence  involves  a  tireless  work  ethic  and  a 
tremendous  devotion  to  the  field.  Although  educators  can  encourage  residents  to 
approach  their  work  with  passion  and  energy,  there  is  no  prescribed  curriculum  that  will 
produce  surgeons  with  these  competencies. 

From  a  psychological  perspective,  the  present  study  showed  that  many  of  the 
characteristics  of  an  outstanding  surgeon  are  personality  related.  A  comparison  of  the 
competency  model  for  surgeons  with  those  from  other  professions  revealed  several 
common  “core”  competencies  (e.g.,  communication  skills,  detail-oriented,  planning,  etc.). 
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In  this  respect,  there  may  be  opportunities  to  include  surgeons  in  future  research  aimed  at 
identifying  common  traits  among  leaders.  On  the  other  hand,  some  of  the  competencies 
for  surgeons  were  unique.  Specifically,  the  top  competency  identified  in  the  present  study 
was  Dedication  to  Patient  Care.  It  is  unlikely  that  one  would  find  this  level  of  dedication 
to  the  “customer”  in  any  other  service  industry.  Thus,  future  researchers  may  be 
interested  in  studying  the  psychological  processes  that  underlie  this  commitment  to 
helping  and  healing  others. 

Finally,  it  should  be  noted  that  surgical  educators  could  benefit  from  1-0 
psychologists  who  have  expertise  in  selection,  training,  and  performance  assessment.  For 
example,  physicians  may  currently  rely  on  standard  questionnaires  developed  for  other 
applications  as  indirect  measures  of  competency.  Industrial-Organizational 
psychologists,  however,  could  develop  measures  that  specifically  assess  competencies  at 
a  behavioral  level.  Regarding  the  present  competency  model,  1-0  psychologists  could 
assist  the  surgical  community  through  the  development  of  reliable  and  valid  tests  that 
measure  personality-related  competencies.  For  example,  since  Integrity  emerged  as  an 
important  competency,  1-0  psychologists  may  wish  to  work  with  subject  matter  experts 
to  develop  and  validate  a  specific  measure  of  integrity  that  can  be  used  to  screen  surgical 
candidates.  Thus,  if  the  requirements  for  the  next  generation  of  surgeons  are  to  reflect  the 
competencies  identified  in  the  present  study,  it  is  likely  that  1-0  psychologists  will  help 
surgery  its  vision. 
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APPENDIX  A 
RECRUITMENT  LETTER 


Dear  Dr. _ , 

My  name  is  Hope  Hanner-Bailey.  I  am  a  graduate  student  in  Industrial-Organizational 
Psychology,  studying  at  Old  Dominion  University  in  Norfolk,  Virginia.  In  my 
dissertation,  I  am  investigating  the  job  of  a  surgeon.  More  specifically,  I  am  looking  at 
the  characteristics  that  distinguish  expert  surgeons  from  average  ones.  The  objective  of 
this  study  is  to  develop  a  thorough  description  of  competencies  that  can  ultimately  be 
used  to  enhance  curriculum  development,  selection,  training,  and  performance  evaluation 
of  surgical  residency  programs. 

You  have  been  identified  by  a  small  group  of  surgeons  at  Eastern  Virginal  Medical 
School  as  an  exceptional  surgeon  in  your  specialty.  I  am  conducting  brief  phone 
interviews  with  several  exceptional  surgeons  from  across  the  country.  The  interview 
consists  of  seven  questions  and  it  will  be  approximately  30  minutes  in  duration.  In  this 
interview,  you  will  be  asked  to  share  your  opinions  regarding  some  of  the  traits,  skills, 
and  abilities  that  combine  to  form  the  truly  exceptional  surgeon. 

Thank  you  very  much  in  advance  for  considering  to  be  interviewed  for  my  dissertation. 

If  you  have  any  questions,  or  if  you  are  willing  to  participate,  please  e-mail  me  at: 
hhanner@odu.edu  and  I  will  work  around  your  schedule  to  find  at  time  that  is  convenient 
for  you.  I  look  forward  to  hearing  from  you. 


Thank  you, 

Hope  Hanner-Bailey 
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APPENDIX  B 

CONTENT  AREAS  AND  COMMENTS  BY  EXPERTS  THAT  FAILED  TO 
MEET  THE  INCLUSION  CRITERION 


Technical  Skill 


1.  An  exceptional  surgeon  has  highly  honed  technical  skills. 

2.  This  surgeon  can  maneuver  difficult  pelvic  areas  and  tumors. 

3.  They  have  great  hand-eye  coordination. 

4.  Technical  part-  relatively  fast  in  the  OR  because  he  has  a  plan  and  he  sticks  to  it. 
Economy  of  motion. 

5.  An  exceptional  surgeon  displays  technical  excellence. 

6.  He  is  technically  excellent. 

7.  They  have  amazing  technical  ability. 

8.  They  are  technically  adroit. 

9.  He  takes  pride  in  any  difficult  operation  that  others  would  have  a  hard  time 
performing. 

Leadership  Skill 

1 .  Being  a  leader-  one  people  go  to  with  problems. 

2.  Being  a  strong  team  leader.  Prefers  to  work  alone  but  also  does  well  in  a  group 
setting. 

3.  Being  a  strong  leader. 

4.  The  third  example  was  a  great  technician  and  a  great  leader.  People  respected 
him  and  wanted  to  emulate  him.  It  is  very  difficult  to  find  all  three 
examples/qualities  in  a  single  surgeon. 

5.  Collaborative;  willing  to  share  the  playing  field  with  others. 

6.  Values  the  opinions  of  everyone  he  works  with,  including  physician  assistants. 
They  often  can  tell  him  how  other  surgeons  in  his  practice  perform  and  offer 
advice  “Dr.  Such  and  Such  does  it  this  way.”  He  wants  to  create  a  climate  where 
everyone’s  input  matters. 

7.  He  remained  a  calm  team  leader.  He  had  the  presence  of  mind  to  fix  the  problem. 

Communication  Skill 


1 .  They  are  good  communicators.  They  communicate  (verbally  and  written)  well 
with  patients  and  other  professionals. 

2.  S/he  must  be  a  strong  communicator. 

3.  They  speak  very  clearly. 

4.  He  has  the  ability  to  communicate  well. 

5.  Communicates  well  with  patients,  particularly  when  dealing  with  complicated 
ideas-  much  more  important  than  technical  skill. 

6.  They  use  appropriate  language;  they  quickly  size  up  patients  to  determine  what 
they  can  understand  and  what  to  tell  them  about  their  illness. 
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7.  Great  ability  to  communicate. 

8.  They  are  emotionally  intelligent  and  good  communicators. 

Ability  to  Maintain  Composure 

1 .  These  surgeons  remain  very  calm  and  patient  under  stress. 

2.  This  surgeon  brought  her  current  OR  in  order.  The  nurses  commented  that 
everything  had  quieted  down  since  her  arrival. 

3.  They  present  a  calm  appearance  during  consultations,  dealing  with  families, 
dealing  with  emergencies,  and  in  the  OR. 

4.  They  exhibit  patience. 

5.  He  is  a  patient  surgeon. 

Critical  Nature 


1 .  They  have  a  critical  nature.  They  critically  evaluate  their  work.  They  are  great  at 
self-evaluation  and  self-analysis. 

2.  They  are  skeptical  of  their  own  research  findings. 

3.  He  is  self-critical. 

4.  Continual  self-improvement-  they  are  never  totally  satisfied. 

5.  They  are  incredible  at  self  analysis;  they  are  self  critical.  They  can  analyze  what 
went  wrong. 

6.  Constant  self-analysis  and  self  improvement  (always  try  to  make  themselves 
better). 

Self-Confident 


1 .  They  have  confidence  in  their  ability  to  do  the  case  at  hand-  although  being  over¬ 
confident  can  get  you  in  trouble. 

2.  They  are  confident  in  general. 

3.  Fearless  -  very  little  scares  them. 

4.  They  are  confident  with  their  skills. 

5.  Courage-  not  boldness  though.  Sometimes  the  most  courageous  option  is  not  to 
intervene. 

6.  Supreme  confidence  to  handle  situations. 

Sense  of  Humor 


1.  Sense  of  humor-  with  residents,  nurses,  patients,  family  members,  etc. 

2.  He  has  a  good  sense  of  humor. 

3.  Good  sense  of  humor/doesn’t  take  himself  too  seriously.  Able  to  laugh  at 
himself. 

4.  They  have  a  sense  of  humor. 
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APPENDIX  C 

CONTENT  AREAS  AND  COMMENTS  BY  NOVICES  THAT  FAILED  TO 
MEET  THE  INCLUSION  CRITERION 


Technical  Skill 


1.  They  are  talented  and  have  very  good  technical  skill. 

2.  Strong  technical  skills  and  decision-making  skills. 

3.  Excellent  technician/brilliant  takes  on  complicated  cases. 

4.  He  has  excellent  technical  skills. 

5.  They  have  good  manual  dexterity. 

Ability  to  Distill  Problems 

1 .  He  has  the  ability  to  reduce  a  problem  to  its  more  basic  components.  Ability  to 
focus  and  discern  crucial  from  periphery. 

2.  Focusing  on  essentials  of  problems. 

3.  They  have  the  amazing  ability  to  decipher  what  is  and  is  not  important;  cut 
through  the  details.  Boil  it  down  or  distill  information 

Devotion  to  Field 


1.  These  surgeons  have  a  sense  of  duty,  a  calling.  They  are  the  marines  of  medicine. 
They  have  an  overwhelming  sense  that  they  are  going  to  get  the  job  done. 

2.  They  have  an  extreme  sense  of  commitment  to  the  field. 

3.  Dedication  to  the  field-  if  they  are  not  doing  surgery,  they  are  not  happy. 

4.  They  are  extremely  dedicated. 

5.  He  is  dedicated  to  the  field. 

Passion  for  Teaching 

1 .  He  was  willing  to  back  up  residents. 

2.  They  enjoy  teaching  medical  students  and  residents. 

3.  They  are  strong  educators/mentors. 

4.  Taking  a  junior  resident  through  a  case  safely  and  confidently.  Helping  them  to 
get  out  of  trouble. 

Awareness  of  Limitations 


1 .  Great  surgeons  are  aware  of  their  limitations. 

2.  They  are  open  to  other  people’s  opinions,  have  no  issue  asking  others  for  help  and 
incorporating  their  advice  into  action. 

3.  Patient  has  unusual  surgical  problem  and  surgeon  wasn’t  sure  how  to  resolve  it. 
Surgeon  explained  available  options  to  the  patient.  He  started  researching  related 
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articles  and  he  implemented  a  procedure  that  was  in  the  article.  He  asked 
residents  and  medical  students  for  their  input  as  well. 

Efficiency 


1 .  An  exceptional  surgeon  does  procedures  very  quickly  and  efficient.  He  or  she 
doesn’t  get  bogged  down. 

2.  They  don’t  take  shortcuts  even  though  their  work  is  time  consuming  and 
technically  demanding.  They  work  quickly. 

3.  Efficiency  in  the  OR.  Knowing  the  proper  speed  to  go;  being  able  to  cut  down  on 
anesthesia. 

4.  They  are  thorough  and  efficient  individuals. 

Team  Leaders 


1 .  They  believe  in  teamwork. 

2.  They  have  the  ability  to  build  coalitions  and  teams. 

3.  They  lead  others  by  example. 

4.  He  has  the  ability  to  build  and  maintain  a  team. 
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ABSTRACT 


A  group  of  medical  students  were  taught  to  perform  a  tube  thoracostomy  on  a 
mannequin  simulator  in  a  traditional  medical  school  setting  and  demonstrate 
competency  in  that  environment.  Immediately  afterward  and  again  four  months 
later,  they  performed  the  procedure  in  a  CAVE  virtual  environment  running  a 
combat  simulation  under  day  and  nighttime  lighting  conditions.  The  results 
showed  that  accuracy  suffered  in  the  simulation.  Participants  also  needed  more 
time  to  perform  the  procedures  under  the  nighttime  conditions.  Further, 
performance  did  not  change  over  the  retention  interval.  The  results  suggest  that 
although  the  surgical  skills  acquired  by  students  in  a  traditional  medical  school 
setting  are  robust,  they  may  still  be  compromised  under  hazardous  conditions. 
More  important,  these  findings  show  that  virtual  environments  can  provide  a  safe 
and  effective  venue  for  military  medical  personnel  to  train  for  dangerous  duty. 


INTRODUCTION 

Simulators  have  been  a  standard  component  of 
military  training  for  many  years  in  a  variety  of 
contexts  including  aviation,  ground  operations, 
weapons  training,  and  decision  making  in  command 
and  control  operations.  Simulation-based  training 
has  been  an  essential  part  of  combat  preparations  to 
give  our  men  and  women  in  uniform  the 
competitive  advantage.  Unfortunately,  the 
availability  of  simulators  to  train  those  who  must 
care  for  the  wounded  has  been  virtually  non¬ 
existent.  Recently,  however,  several  mannequin- 
based  and  virtual  reality-based  medical  simulators 
have  reached  a  level  of  sophistication  allowing 
them  to  be  included  in  medical  school  curricula 
(Dawson,  2002;  Satava,  2001;  Scerbo,  2006). 
Although  many  of  these  simulators  address 
important  fundamental  medical  procedures  (i.e., 
airway  management),  they  were  not  developed  to 
address  the  unique  requirements  of  combat  casualty 
care.  Consequently,  military  medical  personnel 
who  have  been  in  war  often  acknowledge  that  the 
training  they  receive  in  traditional  medical  school 
contexts  does  not  always  transfer  to  combat 
situations  (Miller,  2003). 


Scerbo  et  al.  (2004)  note  that  simulation 
training  can  provide  many  advantages  including 
controlled  environments,  immediate  feedback 
regarding  performance  and  the  opportunity  to 
practice  or  acquire  skills  under  unique  or  rare 
conditions.  Perhaps  the  most  important  advantage 
of  simulation-based  training  is  that  it  permits  one  to 
train  under  conditions  that  would  be  too  dangerous 
in  actual  operational  settings.  Moreover,  virtual 
environments  have  been  shown  to  be  effective 
media  for  training  other  dangerous  jobs  such  as  the 
dismounted  soldier  (Knerr  et  al.,  1998)  and  military 
checkpoint  guards  (Catanzaro  et  al.,  2003). 
Although  this  represents  a  standard  use  of 
simulation  for  training  in  the  military,  it  has  been 
largely  overlooked  in  the  medical  arena.  Toward 
this  end,  the  goal  of  the  present  study  was  to 
examine  the  performance  of  emergency  surgical 
skills  in  a  virtual  environment  (VE)  under  simulated 
combat  conditions.  Specifically,  the  study  was 
designed  to  determine  the  extent  to  which  surgical 
skills  acquired  in  a  traditional  medical  school 
setting  might  be  compromised  in  a  simulated 
combat  scenario. 

Medical  students  were  taught  an  emergency 
surgical  procedure  and  then  performed  the 


procedure  in  a  CAVE  virtual  environment  under 
simulated  combat  conditions  including  visual  and 
auditory  depictions  of  munitions  fire,  gunfire,  and  a 
virtual  sniper  who  shot  at  the  participants.  They 
performed  the  procedure  under  both  day  and 
nighttime  lighting  conditions.  Two  lighting 
conditions  were  included  to  enhance  the  realism  of 
working  under  different  levels  of  workload  and 
stress.  Specifically,  the  nighttime  condition  was 
included  because  the  conditions  for  visibility  might 
be  variable  in  a  military  combat  situation.  The 
students  were  assessed  immediately  after  learning 
the  procedure  and  then  again  four  months  later.  It 
was  expected  that  performance  would  suffer  in  the 
simulation  and  more  so  under  the  night  conditions. 
Further,  these  deficits  were  expected  to  be  greater 
after  the  four-month  interval. 

METHOD 

Participants 

Fifteen  medical  students  (6  males  and  9  females) 
from  the  Eastern  Virginia  Medical  School  in 
Norfolk,  VA  participated  in  the  study.  All  students 
were  in  their  second  or  third  year  of  training  and 
had  no  prior  experience  with  the  thoracostomy 
procedure  or  the  simulators  used  in  this  study.  The 
participants  received  compensation  for  their  time. 

Virtual  Environment  Combat  Simulation 

The  combat  scenario  was  presented  in  a  CAVE 
Automatic  Virtual  Environment.  The  system 
consists  of  two  computers  connected  through  a 
100-mbps  network  switch.  A  4-pipe  SGI®  ONYX® 
2  computer  was  used  for  the  display  and  to  provide 
the  sound.  This  computer  used  MultiGen- 
Paradigm’s  Vega  software  running  on  the  IRIX® 

6.5  operating  system.  The  second  computer  was  an 
SGI®  02  and  served  as  the  main  console  to  launch 
the  application.  The  images  were  presented  with  a 
resolution  of  1024x768  on  three  10x10  ft  walls.  In 
addition,  a  barricade  of  boxes  was  created  within 
the  CAVE.  A  modified  Radio  Shack  invisible  beam 
entry  alert  system  was  fixed  to  the  top  of  the  boxes 
about  3  ft  above  the  ground.  This  sensor  detects 
when  the  trainee’s  head  is  above  the  safe  cover  of 


the  barricade  and  signals  the  virtual  sniper  to 
engage  (see  below). 

The  combat  simulation  depicted  a  small 
town  under  fire  with  one  building  in  flames. 

Combat  was  simulated  using  the  VEGA  special 
effects  module  to  trigger  visual  and  auditory 
explosion  events  and  background  gunfire  at  specific 
times.  The  events  were  timed  to  repeat  at  specific 
intervals  and  the  entire  scenario  ran  in  a  continuous 
loop  until  the  participant  finished  the  procedure. 

The  audio  track  was  created  with  sound 
samples  from  unrestricted  sources  on  the  internet. 
Vocal  utterances  were  monophonic  and  sampled  at 
a  22.1  kHz.  Background  and  other  supplemental 
audio  sounds  included  gunfire,  explosions,  machine 
gun  fire,  and  some  Ml  tank  fire.  The  audio  files 
were  presented  over  two  channels  with  left  and 
right  speakers  placed  at  approximately  45  and  315 
degrees  relative  to  the  participant,  respectively. 

The  speakers  were  mounted  on  speaker  stands  at  an 
elevation  of  approximately  five  feet.  None  of  the 
audio  sounds  exceeded  90dB  during  the  session. 

Day  and  nighttime  conditions  were  created 
using  different  settings  of  the  time-of-day  feature  in 
the  VEGA  software.  The  daytime  conditions 
produced  enough  ambient  illumination  from  the 
CAVE  walls  to  make  the  barricade,  mannequin,  and 
instruments  easily  visible.  Under  the  nighttime 
conditions,  however,  very  little  illumination  was 
provided  by  the  CAVE  walls  forcing  the 
participants  to  perform  the  procedure  in  almost  total 
darkness  except  for  occasional  explosions  that 
provided  temporary  increases  in  illumination. 

Procedure 

All  students  received  a  didactic  session  covering  the 
fundamentals  of  tube  thoracostomy  (chest  tube 
insertion).  This  procedure  requires  the  surgeon  to 
make  a  1  -2  cm  incision  in  the  skin  near  the 
pectoralis  major  muscle,  puncture  the  pleural  space, 
and  guide  a  tube  into  the  opened  pleural  space  to 
permit  drainage  of  fluid.  Participants  were  then 
instructed  to  perform  the  procedure  on  the 
TraumaMan®  System  mannequin  (Simulab,  Inc., 
Seattle,  WA)  used  throughout  the  world  for  surgery 
education  and  approved  for  the  Advanced  Trauma 
Life  Support®  (ATLS)  Surgical  Skills  Practicum  by 
the  American  College  of  Surgeons.  The  mannequin 


is  a  realistic  anatomical  model  of  the  neck,  chest, 
and  abdomen  containing  latex  replaceable  tissue 
components  and  fluid  reservoirs.  Students  worked 
in  teams  of  two  and  were  allowed  approximately  90 
minutes  of  practice  under  the  supervision  of  an 
attending  surgeon  qualified  to  teach  ATLS®.  After 
training,  each  student  was  required  to  perform  a 
successful  procedure  in  less  than  two  minutes  with 
good  tube  placement  (as  determined  by  the 
surgeon)  to  move  on  to  the  simulation  session.  Tube 
placements  were  rated  as  good,  adequate,  or  poor 
for  topographical  location,  angulation,  and  final 
tube  position.  All  participants  met  these  criteria. 

For  the  initial  test  session,  participants 
reported  to  the  CAVE  facility  within  two  hours  of 
training.  All  participants  were  tested  individually. 
The  participants  were  told  to  play  the  role  of  an 
Army  medic  with  a  team  of  soldiers  under  fire  and 
that  they  needed  to  get  to  the  patient  and  perform 
the  procedure  as  quickly  as  possible.  Each  trial 
began  with  the  participant  standing  at  a  point 
marked  on  the  floor.  They  were  told  to  listen  for  a 
call  for  a  medic  and  start  as  soon  as  they  heard  the 
call.  They  were  also  told  not  to  assess  the  need  for 
the  procedure  or  to  anesthetize  the  patient.  When 
they  finished,  they  were  told  to  return  to  the  starting 
mark  on  the  floor. 

In  addition,  they  were  warned  of  a  sniper  in 
one  of  the  nearby  buildings.  They  were  told  to  take 
cover  behind  the  barricade  because  the  sniper 
would  shoot  to  kill  if  he  got  them  in  his  sights.  If 
they  were  fired  upon,  they  would  hear  a  loud  rifle 
shot.  If  the  sniper  missed,  a  message  was  played 
that  warned  them  to  “Get  down.”  If  they  were 
killed,  they  would  hear  the  phrase,  “Hasta  la  vista, 
baby.”  Although  at  that  point  they  were  considered 
dead,  they  were  told  to  finish  the  procedure.  Each 
participant  performed  the  procedure  twice:  once 
under  daylight  and  once  under  nighttime  conditions 
(order  of  conditions  was  counterbalanced  across 
participants).  Four  months  later,  the  participants 
were  contacted  to  come  back  and  perform  a  second 
test,  again  under  day  and  night  conditions. 

RESULTS 

Because  not  all  of  the  participants  returned 
for  the  second  test  session,  data  were  analyzed  for 
the  1 1  participants  who  completed  both  test 


sessions.  A  mixed  2  (day/night  condition)  x  2 
(immediate/long-term  retention  interval)  x  order 
(night-day/day-night)  ANOVA  was  performed  on 
the  completion  time  data.  The  first  two  variables 
were  manipulated  within  subjects  and  the  order  of 
conditions  was  manipulated  between  subjects. 
Mean  completion  times  were  well  under  the  2-min 
training  criterion  (see  Table  1).  A  main  effect  of 


Figure  1 :  Participant  performing  the  procedure 
under  daylight  combat  conditions. 


Table  1 

Mean  Completion  Times  (in  seconds)  for  Immediate  and 
Long-Term  Retention  and  Day/Night  Conditions 
(standard  deviations  in  parentheses). 


Immediate 

Long  Term 

Mean 

Daylight 

76.10(23.00) 

75.45  (32.08) 

76.83 

Nighttime 

91.36(16.05) 

84.55  (12.68) 

88.10 

Mean 

83.98 

80.95 

day/night  condition  was  found  F(l,9)  =  5.17,/K.05, 
suggesting  that  performance  suffered  more  under 
the  nighttime  condition.  No  other  sources  of 


variance  were  statistically  significant.  Thus, 
completion  times  did  not  differ  between  the  first 
and  second  test  sessions. 

The  accuracy  of  the  tube  placements  was 
rated  using  the  same  criteria  as  in  the  training 
session.  The  proportions  of  tube  placements  rated 
“Good”  are  shown  in  Table  2.  Because  all  of  the 
students  had  to  obtain  “Good”  ratings  on  the 
procedure  prior  to  participating  in  the  experiment, 
one-tail  t-tests  were  used  to  examine  the  deviations 
from  the  expected  proportion  of  1 .00.  There  was  a 
significant  decline  in  the  quality  of  the  tube 
placements  under  both  lighting  conditions  for  the 
immediate  retention  test  (p  <  .025).  There  was  no 
significant  difference  between  long-term  test  and 
the  expected  proportion  or  between  the  immediate 
and  long-term  test  (p  >  .05). 

Table  2 

Proportion  of  Tube  Placements  Rated  Good  for 
Immediate  and  Long-Term  Retention  and  Day /Night 
Conditions. 


Immediate 

Long  Term 

Mean 

Daylight 

.61 

.82 

.715 

Nighttime 

.67 

.85 

.76 

Mean 

.64 

.835 

There  were  also  several  instances  in  which 
participants  were  “killed”  by  the  virtual  sniper. 
Most  of  the  participants  who  were  killed  were  shot 
during  the  first  test  session  (3  during  nighttime 
conditions  and  2  during  daylight  conditions)  versus 
the  second  session  (1  under  day  conditions). 
Although  six  participants  were  fired  upon  in  the 
long-term  retention  condition,  only  one  participant 
was  killed.  If  the  scores  from  the  participants  who 
were  killed  are  removed  and  the  data  reanalyzed  for 
those  who  “survived”,  none  of  the  findings  reach 


statistical  significance;  however,  the  overall  pattern 
of  results  for  lighting  conditions  and  retention 
intervals  remains  the  same  except  for  a  5 -sec 
increase  in  overall  completions  times.  This  finding 
suggests  that  the  faster  performers  were  more 
vulnerable. 

DISCUSSION 

The  primary  goal  of  the  present  study  was  to 
examine  how  training  in  a  typical  medical  school 
environment  would  generalize  to  simulated  combat 
conditions  in  a  VE.  The  findings  were  mixed.  On 
one  hand,  the  results  for  completion  times  were 
encouraging.  The  participants  were  fairly  efficient 
at  performing  the  procedure.  Overall,  the  mean 
times  to  complete  the  procedure  were  84  and  81  sec 
for  the  immediate  and  long-term  retention 
conditions,  respectively.  Both  of  these  times  were 
well  below  the  two-minute  criterion  established  for 
their  practice  session.  As  expected,  the  completion 
times  were  affected  by  lighting  conditions. 
Specifically,  the  participants  took  1 1  sec  longer  on 
average  to  perform  the  procedure  under  nighttime 
conditions.  It  is  also  important  to  note  that  when 
participants  were  tested  four  months  later,  their 
completion  times  were  not  significantly  longer  than 
those  from  their  initial  session.  In  fact,  they  were 
slightly  shorter.  These  results  suggest  that 
participants  were  indeed  able  to  retain  the  skills 
necessary  to  perform  the  thoracostomy  procedure 
over  a  four-month  interval  and  that  their 
performances  had  become  stable. 

Conversely,  the  quality  of  the  procedures 
performed  suffered  under  the  simulated  combat 
conditions.  There  were  significantly  fewer  tube 
placements  rated  “Good”  in  the  immediate  retention 
condition.  Although  the  accuracy  levels  were  better 
in  the  long-term  retention  condition,  there  were  still 
tube  placements  not  rated  “Good”  and  therefore 
below  what  was  achieved  during  initial  training  in  a 
typical  medical  school  setting. 

It  is  important  to  note  that  these  results  may 
present  an  optimistic  picture  of  performance.  There 
were  seven  instances  in  which  the  participants 
failed  to  heed  the  warning  shot  and  were  “killed” 
before  they  could  complete  the  procedure. 

Although  participants  were  asked  to  continue  with 
the  procedure  after  having  been  shot  in  this 


simulation,  under  genuine  circumstances  they  might 
have  been  killed  themselves.  A  finding  such  as  this 
is  indeed  troubling  because  it  suggests  a  potential 
loss  of  critical  medical  personnel  in  addition  to 
jeopardizing  the  safety  of  the  patient.  Also,  there 
was  some  evidence  that  the  faster  performers  were 
more  susceptible  to  sniper  fire.  Further,  these 
results  suggest  the  possibility  of  attentional 
narrowing.  Research  has  shown  that  under  stressful 
conditions,  even  well  trained  individuals  can  fixate 
on  a  particular  stimulus  or  strategy  to  the  exclusion 
of  other  potentially  relevant  information  (Dirkin  & 
Hancock,  1984).  Thus,  it  is  possible  that  even  the 
levels  of  stress  created  by  this  simulated  combat 
conditions  caused  some  participants  to  engage  in 
inappropriate  and  potentially  dangerous  behavior 
that  would  likely  be  exacerbated  under  genuine 
combat  conditions. 

To  our  knowledge,  the  present  study 
represents  the  first  time  that  performance  with  a 
standard  mannequin-based  medical  simulator  has 
been  studied  within  a  fully  immersive  VE.  From 
this  perspective,  our  results  show  that  VEs  can  be  a 
valuable  tool  for  military  medical  training  because 
they  provide  a  rich  context  in  which  to  examine 
performance.  Moreover,  VEs  provide  a  safe 
environment  for  studying  performance  under 
simulated  hazardous  conditions.  Virtual 
environments  open  up  the  possibility  of  examining 
a  wider  variety  of  medical  procedures  performed 
under  an  unlimited  number  of  conditions. 
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Transfer  of  Training  Measures 
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Transfer  of  training  paradigms  provide  an  index  of  skill  generalization  and  help 
establish  the  efficacy  of  training  for  performance  in  real-world  settings.  Consequently, 
determining  the  efficacy  of  simulation-based  training  requires  a  comparison  to  some 
standard.  Often,  the  standard  is  based  on  skills  acquired  in  the  operational  environment. 
However,  the  standard  could  also  be  an  alternative  form  of  training.  In  some  industries, 
this  information  is  difficult  to  obtain,  but  in  medicine  training  in  the  operational 
environment  is  still  the  norm. 

Numerous  measures  of  transfer  have  been  described.  To  illustrate  some  of  these 
measures,  hypothetical  data  from  a  typical  training  scenario  are  presented  in  Table  1 . 
Although  performance  and  training  time  metrics  are  used  to  illustrate  the  calculations, 
other  metrics  such  as  the  number  of  trials  needed  to  reach  criterion  or  level  of  proficiency 
could  also  be  used. 

Table  1 .  Hypothetical  Training  Data  for  Calculating  Transfer  Measures 

Training  Time  Initial  Task  Training  Criterion 

Group  with  Simulator _ Time  in  QE  Time  in  OE  Time 

Grp.  C  60  min  20  hrs  30  min 

Grp.  S  10  hrs  45  min  12  hrs  30  min 


Two  groups  of  participants  undergo  training.  Group  C  is  the  control  group  and  trains  in 
the  operational  environment  (OE).  Group  S  trains  with  a  simulator  and  then  transfers  to 
the  OE.  Participants  are  expected  to  be  able  to  attain  a  task  completion  time  of  30 
minutes  in  the  OE.  For  Group  C,  completion  time  on  the  initial  attempt  is  60  min.  They 
require  20  hours  of  training  in  the  OE  to  reach  the  criterion  of  30  min.  Group  S  trains 
with  the  simulator  for  10  hrs  and  then  moves  into  the  OE.  Their  completion  time  on  the 
initial  attempt  is  45  min.  Group  S  continues  to  train  in  the  OE  for  another  1 0  hrs  until 
they  reach  the  30  min  criterion. 

One  of  the  simplest  methods  for  measuring  transfer  represents  a  comparison  of 
training  times  in  the  OE  (Micheli,  1972).  Based  on  the  time  needed  to  reach  criterion 
performance,  the  percent  transfer  is  calculated  as  follows: 


Training  Time  Grp.  C  -  Training  Time  Grp.  S 

.  X  100 

Training  Time  Grp.  C 

Using  data  from  Table  1,  Group  C  required  20  hours  in  the  OE  to  achieve  a  completion 
time  of  30  minutes  and  Group  S  group  required  12  hours  in  the  OE  (after  practicing  on 


the  simulator).  Therefore,  in  this  example  the  simulator  provides  40%  positive  transfer  to 
the  OE. 

Singley  and  Anderson  (1989)  offer  an  alternative  method  for  measuring  transfer. 
Specifically,  this  method  can  be  used  to  compare  training  in  the  OE  with  the  simulator- 
based  training  by  focusing  on  improvements  in  criterion  task  performance.  Their  formula 
is  as  follows: 


Initial  Task  Time  Grp.  C  -  Initial  Task  Time  Grp.  S 

- - -  X  100 

Initial  Task  Time  Grp.  C  -  Criterion  Time  Grp.  C 


Again,  using  data  from  Table  1,  the  Initial  Task  Times  for  Groups  C  and  S  are  60  min 
and  45  min,  respectively.  The  Group  C  Criterion  Time  is  30  min.  Thus,  training  with  the 
simulator  results  in  50%  transfer.  With  this  measure,  values  generally  fall  between  0  and 
100%;  however,  it  also  possible  to  obtain  values  greater  than  100%.  Under  these 
circumstances,  training  with  the  simulator  produces  performance  benefits  greater  than 
those  that  can  be  obtained  in  the  OE.  This  formula  will  also  return  negative  values 
reflecting  negative  transfer  when  training  with  the  simulator  results  in  poorer  initial  task 
performance  than  training  in  the  OE. 

Transfer  Effectiveness  Ratio 

Some  authors  have  expressed  concerns  over  the  percent  transfer  measures  because  they 
do  not  take  into  consideration  the  time  spent  practicing  with  the  simulator  (Roscoe  & 
Williges,  1980).  Consequently,  Povenmire  and  Roscoe  (1973)  suggested  that  transfer 
should  be  measured  with  the  transfer  effectiveness  ratio  (TER).  The  formula  for  TER  is 
as  follows: 


(Grp.  C  Training  Time  in  OE  -  Grp.  S  Training  Time  in  OE) 


Grp.  S  Training  Time  with  Simulator 


Thus,  the  TER  represents  a  ratio  of  the  time  saved  in  training  to  the  time  spent  with  the 
simulator.  Using  the  data  from  Table  1,  the  TER  would  be  (20-12)/l  0  or  .80.  TER  values 
greater  than  1  indicate  that  training  with  the  simulator  is  efficient.  By  contrast,  TER 
values  less  than  1  show  that  the  simulator  introduces  inefficiency. 

It  is  important  to  understand  that  TER  values  less  than  1  do  not  necessarily  mean 
that  training  with  the  simulator  is  inappropriate.  Simulation-based  training  must  be 
viewed  within  the  entire  educational  curriculum  (see  below).  For  any  hazardous  activity 
that  puts  the  trainee  or  patient  at  risk,  the  simulator  may  be  the  safest  method  of  training 
even  if  it  is  inefficient.  In  medicine,  TER  values  less  than  1  may  be  entirely  acceptable 


given  concerns  over  patient  safety.  That  is,  trainees  who  practice  with  simulators  may  be 
allowed  more  time  in  the  OR  to  reach  criterion  if  they  make  fewer  errors. 

There  are  also  economic  factors  to  consider.  Training  with  the  simulator  may  be  less 
costly  than  training  in  the  operational  environment.  Wickens  and  Hollands  (2000)  argue 
that  investment  in  the  simulator  and  the  cost  of  training  may  impact  decisions  over  the 
acceptability  of  TERs  less  than  1.  They  suggest  determining  the  training  cost  ratio  (TCR), 
a  measure  of  training  costs  in  the  OE/divided  by  simulation-based  training  costs.  For 
example,  if  the  cost  of  training  with  a  simulator  is  $  150/hr  and  training  in  the  OR  is 
$300/hr,  the  TCR  would  be  2.  A  TCR  value  greater  than  1  indicates  cost  savings  for 
simulation-based  training.  Wickens  and  Hollands  also  suggest  that  by  multiplying  the 
TCR  by  the  TER,  one  can  obtain  an  index  of  cost  effectiveness.  Again,  using  the  example 
from  above,  a  TER  of  .80  multiplied  by  a  TCR  of  2.0  would  result  in  a  cost  effectiveness 
value  of  1.6.  Values  less  then  1  indicate  that  simulation-based  training  is  not  cost 
effective. 
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Background/Problem 

Wound  debridement  refers  to  the  removal  of  necrotic, 
devitalized,  or  contaminated  tissue  and/or  foreign  material 
to  promote  wound  healing.  Surgical  debridement  uses  sharp 
instruments  to  cut  dead  tissue  from  a  wound  and  it  is  the 
quickest  and  most  efficient  method  of  debridement.  A 
wound  debridement  simulator  [1]  can  ensure  that  a  medical 
trainee  is  competent  prior  to  performing  a  procedure  on  a 
genuine  patient. 

Irrigation  is  performed  at  different  stages  of  debridement  in 
order  to  remove  debris  and  reduce  the  bacteria  count 
through  rinsing  the  wound.  This  paper  presents  a  novel 
approach  for  realistic  irrigation  visualization  based  on 
texture  representations  of  debris.  This  approach  applies 
image  processing  techniques  to  a  series  of  images,  which 
model  the  cleanliness  of  the  wound.  The  active  texture  is 
generated  dynamically  based  on  the  irrigation  state,  location, 
and  range. 

Tools  and  Methods 

Texture  mapping  is  a  standard  and  powerful  rendering 
technique  used  in  modem  computer  graphics  and  it 
enhances  realism  and  details  in  a  scene  with  only  a  modest 
increase  in  computational  complexity.  Texture  mapping  lays 
one  or  more  digital  images  onto  a  surface  or  volume. 

In  the  proposed  approach,  the  wound  cleanliness  is  modeled 
by  a  series  of  images,  simulating  a  variety  of  debris  on  the 
contaminated  wound;  however,  at  any  time,  there  is  only 
one  texture  active  over  the  whole  mesh.  The  active  texture  is 
initialized  to  be  the  dirty  image  and  updated  dynamically  at 
different  stages  of  the  surgical  debridement,  as  a  result  of 
combination  of  several  different  images. 

Denote  the  active  texture  as  T,  and  the  images  representing 
dirty  state  and  clean  state  as  Ij,  Ic,  respectively  (The 
irrigation  is  simplified  into  only  two  states  in  this  abstract). 
The  steps  in  the  irrigation  are  as  follows: 

1 .  At  the  beginning  of  irrigation,  let  T  =  Id- 

2.  During  irrigation,  detect  the  possible  intersection  between 
the  wound  and  the  prolonged  syringe  outline.  Compute 
the  intersection  point  P  and  the  triangle  that  contains  P. 
Assuming  the  three  vertices  of  the  intersected  triangle  has 

geometrical  coordinates  ,  v2 ,  V3  e  K3  and  texture 

coordinates  ,  t2 ,  t3  e  1R2 ,  the  texture  coordinate  tp  of  P 
can  be  computed  as  follows: 


P  V3 

hMa‘ 


where  a,-,  i  =  1,  2,  3,  are  the  barycentric  coordinates  of  P, 
expressed  as  a  convex  combination  of  Vj ,  V2 ,  V3 . 


3.  Assuming  that  the  sizes  of  the  texture  and  the  images 
have  been  normalized  into  the  range  of  [0,  l]  x  [0,  l] ,  the 
texture  T  is  updated  as  follows 

nt)lt_v„sr=(i-c)/rf+c/c. 

where  r  is  the  range  of  irrigation  and  c  is  a  coefficient  that 
determines  the  thoroughness  of  the  irrigation. 

Other  critical  techniques  in  the  irrigation  visualization 
include  utilization  of  OpenSceneGraph  [2],  which  is  a  high- 
performance  3D  graphics  toolkit,  and  the  development  of 
collision  detection  algorithms,  which  is  omitted  in  this 
abstract  due  to  space  constraint. 

Results 

The  irrigation  results  are  shown  in  the  figures  that  follow. 
Fig.  1  shows  a  simplified  dirty  wound  with  some  dirt  on  the 
wound.  Fig.  2  shows  the  triangular  mesh  of  the  thigh  used  in 
the  wound  debridement.  The  wound  is  being  cleaned  in  Fig. 
3;  it  can  be  seen  that  the  boundary  between  the  dirty  area 
and  the  cleaned  area  is  smooth,  eliminating  blocking  or 
mosaicing  artifacts  exhibited  by  our  previous  work.  Fig.  4 
shows  the  wound  after  cleaning,  which  is  ready  for  further 
operations  such  as  cutting  dead  tissue  in  the  wound 
debridement. 


Conclusion/Discussion 

A  novel  approach  for  realistic  irrigation  visualization  in  a 
surgical  wound  debridement  simulator  is  proposed,  utilizing 
image  processing  techniques  in  a  series  of  images  modeling 
the  wound  cleanliness.  The  result  is  highly  realistic  and  the 
algorithm  is  simple  to  implement  with  low  cost. 
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Abstract.  Surgical  simulators  are  an  integration  of  many  models,  capa¬ 
bilities,  and  functions.  Development  of  a  working  simulator  requires  the 
flexibility  to  integrate  various  software  models,  to  support  interoperabil¬ 
ity,  and  facilitate  performance  optimizations.  An  object  oriented  frame¬ 
work  is  devised  to  support  multithreaded  integration  of  simulation,  de¬ 
formation,  and  interaction.  A  demonstration  application  has  been  im¬ 
plemented  in  Java,  leveraging  the  features  that  are  built  into  the  lan¬ 
guage  including  multithreading,  synchronization,  and  serialization.  Fu¬ 
ture  work  includes  expanding  the  capabilities  of  the  framework  with  a 
broader  range  of  model  and  interactive  capabilities. 

Keywords.  Surgical  simulation,  Framework,  Java 


Introduction 

Surgical  simulators  are  interactive  systems  that  model  surgical  procedures  on 
simulated  biological  tissues  along  with  other  processes  relevant  to  surgical  pro¬ 
cedures.  The  simulation  process  is  complicated  by  several  scientific  and  techno¬ 
logical  limitations.  Scientific  limitations  include  an  incomplete  understanding  of 
the  properties  of  biological  tissues  and  processes  as  well  as  identifying  those  as¬ 
pects  of  a  simulated  procedure  most  amenable  to  training  transfer.  Technological 
limitations  include  the  inability  to  provide  real  time  performance  for  models  at 
the  desired  fidelity.  In  addition,  input  devices  may  not  provide  a  satisfying  surro¬ 
gate  that  is  consistent  with  the  manipulations  required  in  a  surgical  procedure. 
In  order  to  provide  grounding  to  manage  these  issues  and  facilitating  research,  a 
flexible  software  architecture  is  necessary  to  support  simulator  system  develop¬ 
ment.  In  light  of  these  issues,  an  object  oriented  framework  is  proposed  to  sup¬ 
port  straightforward  integration  of  the  various  functional  components  composing 
a  surgical  simulator.  The  work  described  here  is  part  of  a  larger  project  [1].  This 
paper  is  organized  in  five  sections  including  an  introduction,  a  summary  of  related 
works,  an  overview  of  the  software  framework,  presentation  of  a  demonstration, 
and  finally  a  summary  and  future  work. 

1  Correspondence  to:  Lee  A.  Belfore,  Dept.  ECE,  Old  Dominion  University.  Tel.:  757-683-3846; 
Fax:  757-683-3220;  E-mail:  lbelfore@odu.edu. 


1.  Related  Work 


Surgical  simulation  holds  the  promise  of  giving  medical  practitioners  the  ability  to 
perform  some  of  their  training  on  simulators,  raising  the  expertise  of  practitioners 
when  they  first  interact  with  patients.  In  Reference  [2],  a  technology  independent 
language  CAML  is  introduced  as  a  foundation  for  medical  simulation  systems. 
Specific  to  the  work  here,  Spring  [3]  offers  a  framework  for  surgical  simulation 
that  includes  many  models  for  surgical  intervention,  a  mass-spring  model  for  soft 
tissue  modeling,  and  distributive  collaborative  functionality.  Finally,  the  GiPSi 
system  [4]  provides  a  framework  with  broad  capabilities  that  organ  level  simula¬ 
tions  of  physical,  functional,  and  spatial  modeling. 

2.  An  Overview  of  the  Software  Framework 

The  software  framework  is  built  upon  an  abstract  class  structure  that  holds  the 
relevant  parameters  and  references  to  support  communication  and  interaction 
among  the  respective  objects.  The  abstract  classes  are  then  extended  to  provide  a 
specific  functionality.  All  objects  fundamentally  share  a  common  link  in  the  form 
of  the  basic  geometry  which  each  act  upon  and  update.  Communicating  this  in¬ 
formation  among  the  objects  and  synchronizing  the  updates  enables  all  to  receive 
the  most  current  information  in  a  consistent  fashion.  Further,  having  different 
objects  in  separate  threads  limits  the  interactions  among  objects  to  the  pure  ge¬ 
ometry  information  and  provides  a  pathway  to  a  coarse  granularity  parallel  im¬ 
plementation  through  the  use  of  threads.  Among  the  geometry  linked  objects  can 
include  the  tissue  simulation,  dynamic  texturing,  collision  detection,  remeshing, 
visual  rendering,  and  haptics  rendering.  Each  of  these  can  function  autonomously, 
each  updating  and  responding  at  its  necessary  inherent  update  rate,  provided 
up-to-date  geometry  is  available.  Figure  1  provides  a  generic  perspective  on  the 
interaction  among  these  threads.  The  wound  debridement  simulation  companion 
paper  [1]  describes  an  application  where  this  software  framework  could  be  used. 

3.  Demonstration  Implementation 

A  prototype  implementation  of  this  framework  has  been  developed  in  Java  using 
the  Java3D  API.  This  platform  was  selected  for  convenience  as  it  includes  all  of 
the  powerful  features  of  Java,  i.e.  object  oriented  class  structure,  multithread¬ 
ing,  serialization,  and  synchronization.  Furthermore,  the  Java  platform  provides 
some  inherent  capabilities  that  can  be  leveraged  in  collaborative  applications. 
Among  the  features  of  the  implementation,  abstract  classes  are  defined  to  hold 
information  that  must  be  synchronized  among  threads.  Abstract  classes  define 
the  generic  features  for  each  major  thread  and  are  extended  to  provide  the  func¬ 
tional  implementations  desired.  Furthermore,  different  functionalities,  i.e.  model 
enhancements,  model  evaluation,  can  be  easily  replaced  and  evaluated  without 
disrupting  the  rest  of  the  application.  Figure  2  shows  an  example  screen  capture 
of  the  demonstration  application. 


Figure  1.  Software  Architecture 


Figure  2.  Example  Image 


4.  Summary  and  Future  Work 

In  this  paper,  we  have  presented  a  software  framework  that  can  be  used  in  the 
development  of  surgical  simulation  virtual  environments.  We  plan  to  continue 
to  develop  the  framework,  to  investigate  its  operation  on  a  C++  platform  for 
enhanced  performance,  and  work  towards  simpler  integration  with  other  libraries 
and  commercial  products. 
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